Re: Moving File API: Directories and System API to Note track?

[Brendan Eich]

Glenn Maynard wrote:
I'm interested in the same from Mozilla side: what are the real issues 
that you think are unsolvable, or do you just think the underlying use 
cases aren't compelling enough for the work required?


Speaking for myself, not for all Mozillans here, I find the use-cases 
Eric U listed two messages ahead of yours in the thread compelling. I 
like the interface Maciej synthesized from prior designs. Filesystems 
are not databases and they have their uses. This is not a difficult 
proposition! Of course, when all you have is a database then everything 
looks like your thumb... :-|


/be

RE: Moving File API: Directories and System API to Note track?

[SULLIVAN, BRYAN L]

> -Original Message-
> From: Olli Pettay [mailto:olli.pet...@helsinki.fi]
> Sent: Thursday, October 04, 2012 3:48 AM
> To: SULLIVAN, BRYAN L
> Cc: Maciej Stachowiak; Glenn Maynard; Eric U; public-webapps@w3.org
> Subject: Re: Moving File API: Directories and System API to Note track?
> 
> On 09/26/2012 01:32 AM, SULLIVAN, BRYAN L wrote:
> > *From:*Maciej Stachowiak [mailto:m...@apple.com]
> > *Sent:* Tuesday, September 25, 2012 2:59 PM
> > *To:* Glenn Maynard
> > *Cc:* Eric U; o...@pettay.fi; public-webapps@w3.org
> > *Subject:* Re: Moving File API: Directories and System API to Note
> track?
> >
> > Hi Glenn,
> >
> > I read over your points. But I don't think they would change Apple's
> calculation about exposing an API to the real user filesystem in Safari,
> > particularly as specified. I do think that my more minimal API might
> also be a better fit for the "real filesystem" use case, as it removes a
> bunch of
> > unnecessary levels of indirection and abstraction that exist in the
> other proposals. But I think it is unlikely we would expose that aspect.
> >
> > We are still open to solving the "sandboxed local storage area" use
> case, with a more minimal API. But first we'd like to understand why
> those use
> > cases can't be solved with targeted additions to IndexedDB (see forked
> thread).
> >
> > Can IndexedDB support the use case of managing a media or document
> library on the local device, in which the size of library can run into
> hundreds or
> > files and gigabytes of storage?
> I don't see any technical problems with that.
> 
Other than hard limits on storage per origin and how the virtual filesystem 
would be organized and managed, neither do I, but I am not an expert in browser 
design, that's why I asked the question. If there are no technical problems, 
anyone have thoughts on how soon we can expect to see browsers that provide a 
virtual filesystem implementation based upon Indexed DB?

> 
> > Local storage of media is one of the key capabilities needed to enable
> HTML5-based offline media players as a
> > realistic option. If the browser can effectively provide a virtual
> filesystem with performance characteristics equivalent to the underlying
> OS, then
> > perhaps IndexedDB may suffice for filesystem access. But which
> browsers intend to support that level of functionality via IndexexDB?
> >
> Isn't that part of IndexedDB, to be able to store files efficiently in
> it. So I would assume all the implementations will
> support that.

I don't think specific efficiency objectives are a normative requirement of 
Indexed DB, but the question was more whether browser vendors would claim that 
their Indexed DB implementation will provide performance equivalent to the 
underlying OS. Apart from the technical challenges, it's unrealistic to use 
Indexed DB for this purpose if the performance cannot match the underlying OS.

Re: Moving File API: Directories and System API to Note track?

[Olli Pettay]

On 09/26/2012 01:32 AM, SULLIVAN, BRYAN L wrote:

*From:*Maciej Stachowiak [mailto:m...@apple.com]
*Sent:* Tuesday, September 25, 2012 2:59 PM
*To:* Glenn Maynard
*Cc:* Eric U; o...@pettay.fi; public-webapps@w3.org
*Subject:* Re: Moving File API: Directories and System API to Note track?

Hi Glenn,

I read over your points. But I don't think they would change Apple's 
calculation about exposing an API to the real user filesystem in Safari,
particularly as specified. I do think that my more minimal API might also be a better fit 
for the "real filesystem" use case, as it removes a bunch of
unnecessary levels of indirection and abstraction that exist in the other 
proposals. But I think it is unlikely we would expose that aspect.

We are still open to solving the "sandboxed local storage area" use case, with 
a more minimal API. But first we'd like to understand why those use
cases can't be solved with targeted additions to IndexedDB (see forked thread).

Can IndexedDB support the use case of managing a media or document library on 
the local device, in which the size of library can run into hundreds or
files and gigabytes of storage?

I don't see any technical problems with that.



Local storage of media is one of the key capabilities needed to enable 
HTML5-based offline media players as a
realistic option. If the browser can effectively provide a virtual filesystem 
with performance characteristics equivalent to the underlying OS, then
perhaps IndexedDB may suffice for filesystem access. But which browsers intend 
to support that level of functionality via IndexexDB?


Isn't that part of IndexedDB, to be able to store files efficiently in it. So I 
would assume all the implementations will
support that.

Re: Sandboxed Filesystem use cases? (was Re: Moving File API: Directories and System API to Note track?)

[Eric U]

Asking about use cases that can be served by a filesystem API, but not
by IDB, is reasonable [and I'll respond to it below], but it misses a
lot of the point.  The users I've talked to like the FS API because
it's a simple interface that everyone already understands, that's
powerful enough to handle a huge variety of use cases.

Sure, the async API makes it a bit more complicated.  Every API that
handles large data is stuck with the same overhead there.  But
underneath that, people know what to expect from it and can figure it
out very quickly.

You just need to store 100KB?
  1) Request a filesystem.
  2) Open a file.
  3) Write your data.

Need a URL for that?  Sure, it's just a file, so obviously that works.

Want it organized in directories just like your server or dev environment?
Go ahead.

You don't have to write SQL queries, learn how to organize data into
noSQL tables, or deal with version change transactions.

If you want to see what's in your data store, you don't need to write
a viewer to dump your tables; you just go to the URL of any directory
in your store and browse around.  Our URLs have a natural structure
that matches the directory tree.  If you add URLs to IDB, with its
free-form key/value arrangement, I don't forsee an immediate natural
mapping that doesn't involve lots of escaping, ugly URLs, and/or
limitations.

On to the use cases:

Things that work well in a sandboxed filesystem that don't work well
in IDB [or any of the other current storage APIs] are those that
involve nontransactional modifications of large blobs of data.  For
example, video/photo/audio editing, which involve data that's too big
to store lots of extra copies of for rollback of failed transactions,
and which you don't necessarily want to try to fit into memory.
Overwriting just the ID3 tag of an MP3, or just the comment section of
the EXIF in a JPEG, would be much more efficient via a filesystem
interface.  Larger series of modifications to those files, which you
don't want to hold in memory, would be similar.

I know Jonas wants to bolt nontransactional data onto the side of IDB
via FileHandle, but I think that the cure there is far worse than the
disease, and I don't think anyone at Google likes that proposal.  I
haven't polled everyone, but that's the impression I get.

Beyond individual use cases:

When looking at use cases for a filesystem API, people often want to
separate the sandboxed cases and the non-sandboxed cases ["My Photos",
etc.].  It's also worthwhile to look at the added value of having a
single API that works for both cases.  You have a photo organizer that
works in the sandbox with downloaded files?  If your browser supports
external filesystems, you can adapt your code to run in either place
with a very small change [mainly dealing with paths that aren't legal
on the local system].  If you're using IDB in the sandbox, and have a
different API to expose media directories, you've got to start over,
and then you have to maintain both systems.

One added API?

It's pretty clear that people see the value of an API that lets one
access "My Photos" from the web.  That API is necessarily going to
cope with files and directories on some platforms, even if others
don't expose directories as such.  If we're going to need to add a
filesystem API of some kind to deal with that, also using the same API
to manage a sandboxed storage area seems like a very small addition to
the web platform, unlike the other storage APIs we've added in the
past.


Regarding your final note:  I'm not sure what you're talking about
with BlobBuilder; is that the EXIF overwrite case you're trying to
handle?  If so, File[Handle|Writer] with BlobBuilder and seek seems to
handle it better than anything else.

Eric

On Tue, Sep 25, 2012 at 11:57 AM, Maciej Stachowiak  wrote:
>
> On Sep 25, 2012, at 10:20 AM, James Graham  wrote:
>
>>
>> In addition, this would be the fourth storage API that we have tried to 
>> introduce to the platform in 5 years (localStorage, WebSQL, IndexedDB being 
>> the other three), and the fifth in total. Of the four APIs excluding this 
>> one, one has failed over interoperability concerns (WebSQL), one has 
>> significant performance issues and is discouraged from production use 
>> (localStorage) and one suffers from a significant problems due to its legacy 
>> design (cookies). The remaining API (IndexedDB) has not yet achieved 
>> widespread use. It seems to me that we don't have a great track record in 
>> this area, and rushing to add yet another API probably isn't wise. I would 
>> rather see JS-level implementations of a filesystem-like API on top of 
>> IndexedDB in order to work out the kinks without creating a legacy that has 
>> to be maintained for back-compat than native implementations at this time.
>
> I share your concerns about adding yet-another-storage API. (Although I 
> believe there are major websites that have adopted or are in the process of 
> adopting IndexedDB). I like m

RE: Moving File API: Directories and System API to Note track?

[SULLIVAN, BRYAN L]

From: Maciej Stachowiak [mailto:m...@apple.com]
Sent: Tuesday, September 25, 2012 2:59 PM
To: Glenn Maynard
Cc: Eric U; o...@pettay.fi; public-webapps@w3.org
Subject: Re: Moving File API: Directories and System API to Note track?


Hi Glenn,

I read over your points. But I don't think they would change Apple's 
calculation about exposing an API to the real user filesystem in Safari, 
particularly as specified. I do think that my more minimal API might also be a 
better fit for the "real filesystem" use case, as it removes a bunch of 
unnecessary levels of indirection and abstraction that exist in the other 
proposals. But I think it is unlikely we would expose that aspect.

We are still open to solving the "sandboxed local storage area" use case, with 
a more minimal API. But first we'd like to understand why those use cases can't 
be solved with targeted additions to IndexedDB (see forked thread).

Can IndexedDB support the use case of managing a media or document library on 
the local device, in which the size of library can run into hundreds or files 
and gigabytes of storage? Local storage of media is one of the key capabilities 
needed to enable HTML5-based offline media players as a realistic option. If 
the browser can effectively provide a virtual filesystem with performance 
characteristics equivalent to the underlying OS, then perhaps IndexedDB may 
suffice for filesystem access. But which browsers intend to support that level 
of functionality via IndexexDB?

Re: Moving File API: Directories and System API to Note track?

[Glenn Maynard]

On Tue, Sep 25, 2012 at 4:59 PM, Maciej Stachowiak  wrote:

> I read over your points. But I don't think they would change Apple's
> calculation about exposing an API to the real user filesystem in Safari,
> particularly as specified. I do think that my more minimal API might also
> be a better fit for the "real filesystem" use case, as it removes a bunch
> of unnecessary levels of indirection and abstraction that exist in the
> other proposals. But I think it is unlikely we would expose that aspect.
>

I don't personally care either way about which style of API is used, as
long as it's designed carefully and satisfies the use cases.

But I don't understand why this feature is being rejected on its face.  No
issues have been raised that seem unsolvable; when solutions are presented
to issues, the response is "interesting, but still no", so it feels like
this is being rejected without any explanation at all.

I'm interested in the same from Mozilla side: what are the real issues that
you think are unsolvable, or do you just think the underlying use cases
aren't compelling enough for the work required?

-- 
Glenn Maynard

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

Hi Glenn,

I read over your points. But I don't think they would change Apple's 
calculation about exposing an API to the real user filesystem in Safari, 
particularly as specified. I do think that my more minimal API might also be a 
better fit for the "real filesystem" use case, as it removes a bunch of 
unnecessary levels of indirection and abstraction that exist in the other 
proposals. But I think it is unlikely we would expose that aspect.

We are still open to solving the "sandboxed local storage area" use case, with 
a more minimal API. But first we'd like to understand why those use cases can't 
be solved with targeted additions to IndexedDB (see forked thread).

Regards,
Maciej

On Sep 25, 2012, at 12:35 PM, Glenn Maynard  wrote:

> On Wed, Sep 19, 2012 at 3:46 PM, James Graham  wrote:
> Indeed. We are not enthusiastic about implementing an API that has to 
> traverse directory trees as this has significant technical challenges, or may 
> expose user's path names, as this has security implications. Also AIUI this 
> API is not a good fit for all platforms.
> 
> I don't think there are any unsolvable problems for traversing native 
> directory trees.  (For example, hard links can be dealt with if they do turn 
> up--keep track of inodes in parents--and pathname limitations put an upper 
> limit, preventing infinite recursion.  I think they're very unlikely in 
> practice--infinite directory loops would break tons of native apps, too.  No 
> filesystem I've tried in Linux allows it.)  I don't think any proposals 
> expose user pathnames (eg. above the item that was dropped/opened).
> 
> The main issue that needs to be explored is how to prevent users from 
> accidentally giving more access than they mean to, eg. pages saying "please 
> open C:\ and drag in Users", and users not realizing what they're doing.  
> This is critically important, but I think it's too early to conclude that 
> this is unsolvable.
> 
> (That's a UI issue more than a technical issue.  There are other clearly 
> technical issues, but I haven't seen any raised that look unsolvable.  The 
> two tricky ones I've seen are maximum filename/path length limitations, and 
> valid characters varying between platforms, which have been discussed a lot 
> and would need to be revisited--it's been too long and I forget where that 
> left off.)
> 
> 
> On Fri, Sep 21, 2012 at 7:37 PM, Maciej Stachowiak  wrote:
> - I'm not keen on exposing portions of the user's filesystem. In particular:
> - This seems like a potential security/social-engineering risk.
> - This use case seems more relevant to system apps based on Web 
> technology than Web apps as such; I fear that system-app-motivated complexity 
> is bloating the api.
> 
> These are the *primary* use cases of a filesystem-like API.  There's lot of 
> value in being able to let a media player app play and (optionally) 
> manipulate my local media directories, for example--obviously I won't move my 
> music into IndexedDB.
> 
> (I don't find the sandboxed case very compelling, and while I consider 
> IndexedDB "to-be-proven" for things like storing gigabytes of data for a 
> game, it should definitely be given that chance before introducing another 
> API for that set of use cases.)
> 
> - Many of Apple's key target platforms don't even *have* a user-visible 
> filesystem.
> 
> It's not inconsistent to have APIs that aren't relevant on every single 
> platform.  When you don't have a user filesystem in the first place, the use 
> cases themselves go away and you don't care about any of this.
> 
> (For the sandboxed case, this isn't relevant.  It would be useful to separate 
> the sandboxed and native discussions, since except for the API style 
> question, the issues and objections are almost completely distinct, and it's 
> sometimes hard to keep track of which one we're talking about.)
> 
> - We already have way too many storage APIs. To add another, the use cases 
> would have to be overwhelmingly compelling, and highly impractical to serve 
> by extending any of the other storage APIs (e.g. IndexedDB).
> 
> I do find being able to allow users to work with their native data, just as 
> native apps do, to be overwhelmingly compelling.  I have about 4 TB of data, 
> and the only way I can use any of it with web apps is by dragging in 
> individual, flat lists of files, and even that's a strictly one-way street 
> (FileSaver is only a very tiny help here).
> 
> -- 
> Glenn Maynard
> 
> 

Re: Moving File API: Directories and System API to Note track?

[Glenn Maynard]

On Wed, Sep 19, 2012 at 3:46 PM, James Graham  wrote:

> Indeed. We are not enthusiastic about implementing an API that has to
> traverse directory trees as this has significant technical challenges, or
> may expose user's path names, as this has security implications. Also AIUI
> this API is not a good fit for all platforms.
>

I don't think there are any unsolvable problems for traversing native
directory trees.  (For example, hard links can be dealt with if they do
turn up--keep track of inodes in parents--and pathname limitations put an
upper limit, preventing infinite recursion.  I think they're very unlikely
in practice--infinite directory loops would break tons of native apps,
too.  No filesystem I've tried in Linux allows it.)  I don't think any
proposals expose user pathnames (eg. above the item that was
dropped/opened).

The main issue that needs to be explored is how to prevent users from
accidentally giving more access than they mean to, eg. pages saying "please
open C:\ and drag in Users", and users not realizing what they're doing.
This is critically important, but I think it's too early to conclude that
this is unsolvable.

(That's a UI issue more than a technical issue.  There are other clearly
technical issues, but I haven't seen any raised that look unsolvable.  The
two tricky ones I've seen are maximum filename/path length limitations, and
valid characters varying between platforms, which have been discussed a lot
and would need to be revisited--it's been too long and I forget where that
left off.)


On Fri, Sep 21, 2012 at 7:37 PM, Maciej Stachowiak  wrote:

> - I'm not keen on exposing portions of the user's filesystem. In
> particular:
> - This seems like a potential security/social-engineering risk.
> - This use case seems more relevant to system apps based on Web
> technology than Web apps as such; I fear that system-app-motivated
> complexity is bloating the api.
>

These are the *primary* use cases of a filesystem-like API.  There's lot of
value in being able to let a media player app play and (optionally)
manipulate my local media directories, for example--obviously I won't move
my music into IndexedDB.

(I don't find the sandboxed case very compelling, and while I consider
IndexedDB "to-be-proven" for things like storing gigabytes of data for a
game, it should definitely be given that chance before introducing another
API for that set of use cases.)

- Many of Apple's key target platforms don't even *have* a user-visible
> filesystem.
>

It's not inconsistent to have APIs that aren't relevant on every single
platform.  When you don't have a user filesystem in the first place, the
use cases themselves go away and you don't care about any of this.

(For the sandboxed case, this isn't relevant.  It would be useful to
separate the sandboxed and native discussions, since except for the API
style question, the issues and objections are almost completely distinct,
and it's sometimes hard to keep track of which one we're talking about.)

- We already have way too many storage APIs. To add another, the use cases
> would have to be overwhelmingly compelling, and highly impractical to serve
> by extending any of the other storage APIs (e.g. IndexedDB).
>

I do find being able to allow users to work with their native data, just as
native apps do, to be overwhelmingly compelling.  I have about 4 TB of
data, and the only way I can use any of it with web apps is by dragging in
individual, flat lists of files, and even that's a strictly one-way street
(FileSaver is only a very tiny help here).

-- 
Glenn Maynard

Sandboxed Filesystem use cases? (was Re: Moving File API: Directories and System API to Note track?)

[Maciej Stachowiak]

On Sep 25, 2012, at 10:20 AM, James Graham  wrote:

> 
> In addition, this would be the fourth storage API that we have tried to 
> introduce to the platform in 5 years (localStorage, WebSQL, IndexedDB being 
> the other three), and the fifth in total. Of the four APIs excluding this 
> one, one has failed over interoperability concerns (WebSQL), one has 
> significant performance issues and is discouraged from production use 
> (localStorage) and one suffers from a significant problems due to its legacy 
> design (cookies). The remaining API (IndexedDB) has not yet achieved 
> widespread use. It seems to me that we don't have a great track record in 
> this area, and rushing to add yet another API probably isn't wise. I would 
> rather see JS-level implementations of a filesystem-like API on top of 
> IndexedDB in order to work out the kinks without creating a legacy that has 
> to be maintained for back-compat than native implementations at this time.

I share your concerns about adding yet-another-storage API. (Although I believe 
there are major websites that have adopted or are in the process of adopting 
IndexedDB). I like my version better than the Google one, too, but I also worry 
about whether we should be adding another storage API at all.

I think we need to go back to the use case for sandboxed filesystem storage and 
understand which use cases cannot be served with IndexedDB.


Here are some use cases I have heard:

(1) A webapp (possibly working on offline mode) wants to stage files for later 
upload (e.g. via XHR).
Requirements:
- Must be able to store distinct named items containing arbitrary 
binary data.
- Must be able to read the data back for later upload.
- Must be able to delete items.

(2) A web-based mail client wants to download the user's attachments locally, 
then reference them by URL from the email and allow them to be extracted into 
the user's filesystem space.
Requirements:
- Must be able to store distinct named items containing arbitrary 
binary data.
- Must be able to reference items by persistent  URL from constructs in 
a webpage that use URLs.
- Must be able to delete items.

(3) A web-based web developer tool downloads copies of all the resources of a 
webpage, lets the user edit the webpage live potentially adding new resources, 
and then uploads it all again to one or more servers.
Requirements: 
- Must be able to store distinct named items containing arbitrary 
binary data.
- Must be able to replace items.
- Must be able to reference items by persistent  URL from constructs in 
a webpage that use URLs.
- Must be able to delete items.
- Must be able to enumerate items.
Highly desirable:
- Hierarchical namespace.

(4) A game wants to download game resources locally for efficient operation, 
and later update them
Requirements: 
- Must be able to store distinct named items containing arbitrary 
binary data.
- Must be able to replace items.
- Must be able to reference items by persistent URL from constructs in 
a webpage that use URLs.
- Must be able to delete items.
Highly desirable:
- Hierarchical namespace.


I believe the only requirement here that is not met by IndexedDB is:
- The ability to reference an item by persistent URL.

IndexedDB has enumeration, hierarchical namespace, ability to add, replace, 
remove, get, etc.


Are there other use cases? In particular, are there use cases that justify a 
whole new storage API instead of adding this one feature to IndexedDB?


Note: one aspect of the MinimalFileSystem proposal that is not obviously 
required by any of these use cases is the ability to incrementally update a 
file (beyond what you could already do with slice() and BlobBuilder). Basically 
the whole FileHandle interface. Is there truly a use case that you can't 
satisfy by using BlobBuilder to make your update and then atomically replacing?


Regards,
Maciej

Re: Moving File API: Directories and System API to Note track?

[James Graham]

On Tue, 25 Sep 2012, Brendan Eich wrote:


Maciej Stachowiak wrote:

On Sep 22, 2012, at 9:35 PM, Maciej Stachowiak  wrote:


On Sep 22, 2012, at 8:18 PM, Brendan Eich  wrote:


And two of the interfaces are generic and reusable in other contexts.
Nice, and DOMRequest predates yours -- should it be done separately since 
(I believe) it is being used by other proposals unrelated to 
FileSystem-like ones?


Sorry if I missed it and it's already being split out.
Yes, I borrowed DOMRequest. I think DOMRequest and DOMMultiRequest could 
be a separate spec, if that sort of asynchronous response pattern is 
generally useful. And it seems like it might be. That would leave only two 
interfaces specific to the Minimal File System proposal, Directory and 
FileHandle.


Here's an alternate version where I renamed some things to match Filesystem 
API and FileWriter, and added the missing key feature of getting a 
persistent URL for a file in a local filesystem (extending the URL 
interface). It's still a much simpler API that provides most of the same 
functionality.


https://trac.webkit.org/wiki/MinimalFileStorageAlternate


Even better.

What's the next step? I would hope we can make device storage be this, but 
that's just old Unix-hacker me (yes, no pathnames, you have to namei by 
calling get for each component -- that's a feature).


*Personally* I like this proposal more than the original Google one, but 
I'm still have concerns. Every operation requiring a callback seems likely 
to lead to a "pyramid of doom" and unmaintainable code. Since this seems 
to be an increasing problem with certain classes of JS APIs (the SQL API 
had similar issues and I wouldn't be surprised if IndexedDB has too), it 
seems like we should work out some solution, either at the langauge level 
or at the DOM level, for managing this complexity rather than adding new 
features that require it. Obviously workers and a sync API are one option, 
but having to keep a worker around just to do storage operations also 
seems rather burdensome.


In addition, this would be the fourth storage API that we have tried to 
introduce to the platform in 5 years (localStorage, WebSQL, IndexedDB 
being the other three), and the fifth in total. Of the four APIs excluding 
this one, one has failed over interoperability concerns (WebSQL), one has 
significant performance issues and is discouraged from production use 
(localStorage) and one suffers from a significant problems due to its 
legacy design (cookies). The remaining API (IndexedDB) has not yet 
achieved widespread use. It seems to me that we don't have a great track 
record in this area, and rushing to add yet another API probably isn't 
wise. I would rather see JS-level implementations of a filesystem-like API 
on top of IndexedDB in order to work out the kinks without creating a 
legacy that has to be maintained for back-compat than native 
implementations at this time.

Re: Moving File API: Directories and System API to Note track?

[Brendan Eich]

Maciej Stachowiak wrote:

On Sep 22, 2012, at 9:35 PM, Maciej Stachowiak  wrote:


On Sep 22, 2012, at 8:18 PM, Brendan Eich  wrote:


And two of the interfaces are generic and reusable in other contexts.

Nice, and DOMRequest predates yours -- should it be done separately since (I 
believe) it is being used by other proposals unrelated to FileSystem-like ones?

Sorry if I missed it and it's already being split out.

Yes, I borrowed DOMRequest. I think DOMRequest and DOMMultiRequest could be a 
separate spec, if that sort of asynchronous response pattern is generally 
useful. And it seems like it might be. That would leave only two interfaces 
specific to the Minimal File System proposal, Directory and FileHandle.


Here's an alternate version where I renamed some things to match Filesystem API 
and FileWriter, and added the missing key feature of getting a persistent URL 
for a file in a local filesystem (extending the URL interface). It's still a 
much simpler API that provides most of the same functionality.

https://trac.webkit.org/wiki/MinimalFileStorageAlternate


Even better.

What's the next step? I would hope we can make device storage be this, 
but that's just old Unix-hacker me (yes, no pathnames, you have to namei 
by calling get for each component -- that's a feature).


/be

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

On Sep 22, 2012, at 9:35 PM, Maciej Stachowiak  wrote:

> 
> On Sep 22, 2012, at 8:18 PM, Brendan Eich  wrote:
> 
>> 
>>> And two of the interfaces are generic and reusable in other contexts.
>> 
>> Nice, and DOMRequest predates yours -- should it be done separately since (I 
>> believe) it is being used by other proposals unrelated to FileSystem-like 
>> ones?
>> 
>> Sorry if I missed it and it's already being split out.
> 
> Yes, I borrowed DOMRequest. I think DOMRequest and DOMMultiRequest could be a 
> separate spec, if that sort of asynchronous response pattern is generally 
> useful. And it seems like it might be. That would leave only two interfaces 
> specific to the Minimal File System proposal, Directory and FileHandle.

Here's an alternate version where I renamed some things to match Filesystem API 
and FileWriter, and added the missing key feature of getting a persistent URL 
for a file in a local filesystem (extending the URL interface). It's still a 
much simpler API that provides most of the same functionality. 

https://trac.webkit.org/wiki/MinimalFileStorageAlternate

Regards,
Maciej

Re: Moving File API: Directories and System API to Note track?

[Kinuko Yasuda]

On Sat, Sep 22, 2012 at 8:32 AM, Eric U  wrote:

> While I don't see any other browsers showing interest in implementing
> the FileSystem API as currently specced,


Just for the record, Blackberry, Tizen/EFL and Netfront seem to have some
support of FileSystem API.
EFL has also uploaded a new FileSystem API patch just a few days ago.
http://html5test.com/compare/feature/files-fileSystem.html

PhoneGap's File API is mostly compatible to the current FileSystem API.
http://docs.phonegap.com/en/1.5.0/phonegap_file_file.md.html#File


I do see Firefox coming
> around to the belief that a filesystem-style API is a good thing,
> hence their DeviceStorage API.  Rather than scrap the API that we've
> put 2 years of discussion and work into, why not work with us to
> evolve it to something you'd like more?  If you have objections to
> specific attributes of the API, wouldn't it be more efficient to
> change just those things than to start over from scratch?  Or worse,
> to have the Chrome filesystem API, the Firefox filesystem API, etc.?
>
> If I understand correctly, folks at Mozilla think having a directory
> abstraction is too heavy-weight, and would prefer users to slice and
> dice paths by hand.  OK, that's a small change, and the
> functionality's roughly equivalent.  We could probably even make
> migration fairly easy with a small polyfill.
>
> Jonas suggests FileHandle to replace FileWriter.  That's clearly not a
> move to greater simplicity, and no polyfill is possible, but it does
> open up the potential for higher perfomance, especially in a
> multi-process browser.  As i said when you proposed it, I'm
> interested, and we'd also like to solve the locking use cases.
>
> Let's talk about it, rather than throw the baby out with the bathwater.
>
> Eric
>
> On Tue, Sep 18, 2012 at 4:04 AM, Olli Pettay 
> wrote:
> > Hi all,
> >
> >
> > I think we should discuss about moving File API: Directories and System
> API
> > from Recommendation track to Note. Mainly because the API hasn't been
> widely
> > accepted nor
> > implemented and also because there are other proposals which handle the
> same
> > use cases.
> > The problem with keeping the API in recommendation track is that people
> > outside
> > standardization world think that the API is the one which all the
> browsers
> > will implement and
> > as of now that doesn't seem likely.
> >
> >
> >
> >
> > -Olli
> >
>
>

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

On Sep 22, 2012, at 8:18 PM, Brendan Eich  wrote:

> 
>> And two of the interfaces are generic and reusable in other contexts.
> 
> Nice, and DOMRequest predates yours -- should it be done separately since (I 
> believe) it is being used by other proposals unrelated to FileSystem-like 
> ones?
> 
> Sorry if I missed it and it's already being split out.

Yes, I borrowed DOMRequest. I think DOMRequest and DOMMultiRequest could be a 
separate spec, if that sort of asynchronous response pattern is generally 
useful. And it seems like it might be. That would leave only two interfaces 
specific to the Minimal File System proposal, Directory and FileHandle.

Regards,
Maciej

Re: Moving File API: Directories and System API to Note track?

[Brendan Eich]

Maciej Stachowiak wrote:

On Sep 21, 2012, at 10:10 PM, Jonas Sicking  wrote:


For what it's worth, I put together a draft for what an API would look
like that has basically the same feature set as the current FileSystem
API, but based on DeviceStorage. It's a much smaller API that the
current FileSystem drafts, but supports things like shallow as well as
deep directory iteration.

https://wiki.mozilla.org/WebAPI/DeviceStorageAPI2

I think that if we at mozilla were to implement a sandboxed
filesystem, it'd be something more like this.


I took a crack at a pruned and cleaned up version:

https://trac.webkit.org/wiki/MinimalFileStorage

Only 4 interfaces (excluding the Navigator addition), 16 methods, 9 attributes. 



Nice!


And two of the interfaces are generic and reusable in other contexts.


Nice, and DOMRequest predates yours -- should it be done separately 
since (I believe) it is being used by other proposals unrelated to 
FileSystem-like ones?


Sorry if I missed it and it's already being split out.


It removes cruft but adds features relative to the Mozilla version:

- Atomic create/open operation


As an old Unix hacker, this warms my heart. (I ignore the camelCaps 
method multiplication to avoid O_ flags, and the spelled-out creatE ;-)



- Renaming files and directories


Atomic rename, must have.


- A Directory interface for less weirdness in directory handling


+1

- Ability to open files in append mode (this makes more sense than an append() operation on the handle, given the way underlying filesystems work) 



Yup.


Features omitted include:
  - Multiple named filesystems (just use directories)


Exactly. Do we still call them mount points?


- Separate enumeration for writing (just use open)


Yup.

- Ability to fetch matadata (does not appear to be needed for the use case) 



Whatsa mata with matadata? :-P

Ok, defer. Fine by me unless Sicking has a pressing use-case to articulate.

It could be simplified a little bit, but really not that much, by removing hierarchy. 



Hierarchy is much of the point here.

I like the way this is going.

I'm curious how things look in hindsight to Google folks. Was the whole 
of FileSystem truly needed? Even if "whole" properly excludes the 
synchronous APIs?


/be

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

On Sep 21, 2012, at 10:10 PM, Jonas Sicking  wrote:

> 
> For what it's worth, I put together a draft for what an API would look
> like that has basically the same feature set as the current FileSystem
> API, but based on DeviceStorage. It's a much smaller API that the
> current FileSystem drafts, but supports things like shallow as well as
> deep directory iteration.
> 
> https://wiki.mozilla.org/WebAPI/DeviceStorageAPI2
> 
> I think that if we at mozilla were to implement a sandboxed
> filesystem, it'd be something more like this.

I took a crack at a pruned and cleaned up version:

https://trac.webkit.org/wiki/MinimalFileStorage

Only 4 interfaces (excluding the Navigator addition), 16 methods, 9 attributes. 
And two of the interfaces are generic and reusable in other contexts.

It removes cruft but adds features relative to the Mozilla version:

- Atomic create/open operation
- Renaming files and directories
- A Directory interface for less weirdness in directory handling
- Ability to open files in append mode (this makes more sense than an append() 
operation on the handle, given the way underlying filesystems work)

Features omitted include:
 - Multiple named filesystems (just use directories)
- Separate enumeration for writing (just use open)
- Ability to fetch matadata (does not appear to be needed for the use case)


It could be simplified a little bit, but really not that much, by removing 
hierarchy.

Regards,
Maciej 

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

What does getMetadata a synchronously return?

I think this API as written is still a fair bit more complex than needed for 
the sandboxed storage use case. It does seem simpler than Filesystem API.

Regards,
Maciej

On Sep 21, 2012, at 10:10 PM, Jonas Sicking  wrote:

> On Fri, Sep 21, 2012 at 5:37 PM, Maciej Stachowiak  wrote:
>> 
>> My personal objections (ones that I think are shared by at least some other 
>> Safari folks):
>> 
>> - It's way too complicated. (As one crude metric, I count 22 interfaces; and 
>> yes, I know many of those are callback interfaces or sync versions of 
>> interfaces; it still seems overengineered).
>> - I see value in the use case of a private sandboxed storage area, as 
>> staging for uploads, to hold downloaded resources, etc. But:
>>- It's not totally clear to me why a filesystemlike API is the best way 
>> to serve these use cases, particularly if in practice it is actually backed 
>> by a database.
>>- The API as designed is way too complex for just that use case.
>> - I'm not keen on exposing portions of the user's filesystem. In particular:
>>- This seems like a potential security/social-engineering risk.
>>- This use case seems more relevant to system apps based on Web 
>> technology than Web apps as such; I fear that system-app-motivated 
>> complexity is bloating the api.
>>- Many of Apple's key target platforms don't even *have* a user-visible 
>> filesystem.
>>- We'd like to keep Web APIs consistent between our various platforms 
>> consistent as much as possible.
>>- I think trying to serve the "real filesystem" use cases adds a lot of 
>> complexity over the "private storage area" use case.
>> - We already have way too many storage APIs. To add another, the use cases 
>> would have to be overwhelmingly compelling, and highly impractical to serve 
>> by extending any of the other storage APIs (e.g. IndexedDB).
>>- In particular, I have heard an explanation of why IndexedDB as it 
>> currently exists can't handle all the use cases for file-like storage, but I 
>> have heard no explanation of why it can't be extended in that direction.
>> 
>> For these reasons, I think it is unlikely that Safari would ever support 
>> Filesystem API in its current form. I could imagine considering a *much* 
>> narrower API, scoped only to the use case of private storage areas for Web 
>> apps, but only if a compelling case is made that there's no other way to 
>> serve that use case.
>> 
>> For what it's worth, even the DeviceStorage API proposal is too complex for 
>> my tastes, in its current iteration.
>> 
>> I think keeping the Filseystem API on the REC track in its current form is 
>> actively bad, because it leads outside observers to be misled about where 
>> the Web platform is going. For example,  sites like  
>> give out "points" for it even though it seems unlikely to advance on the 
>> standards track as it stands today.
> 
> For what it's worth, I put together a draft for what an API would look
> like that has basically the same feature set as the current FileSystem
> API, but based on DeviceStorage. It's a much smaller API that the
> current FileSystem drafts, but supports things like shallow as well as
> deep directory iteration.
> 
> https://wiki.mozilla.org/WebAPI/DeviceStorageAPI2
> 
> I think that if we at mozilla were to implement a sandboxed
> filesystem, it'd be something more like this.
> 
> The FileHandle part of the API is definitely more complex to implement
> than the FileWriter API, but I'd argue that it's actually easier to
> use. For example you can start multiple write operations without
> having to wait for each individual write to finish.
> 
> I also added an example of what a read-only DeviceStorage would look
> like if we wanted something like that for ,
> drag'n'drop or a zip-reader. This shouldn't be considered as part of
> the remaining draft though since exposing a filesystem in those
> scenarios might very well be overkill.
> 
> / Jonas

Re: Moving File API: Directories and System API to Note track?

[Jonas Sicking]

On Fri, Sep 21, 2012 at 5:37 PM, Maciej Stachowiak  wrote:
>
> My personal objections (ones that I think are shared by at least some other 
> Safari folks):
>
> - It's way too complicated. (As one crude metric, I count 22 interfaces; and 
> yes, I know many of those are callback interfaces or sync versions of 
> interfaces; it still seems overengineered).
> - I see value in the use case of a private sandboxed storage area, as staging 
> for uploads, to hold downloaded resources, etc. But:
> - It's not totally clear to me why a filesystemlike API is the best way 
> to serve these use cases, particularly if in practice it is actually backed 
> by a database.
> - The API as designed is way too complex for just that use case.
> - I'm not keen on exposing portions of the user's filesystem. In particular:
> - This seems like a potential security/social-engineering risk.
> - This use case seems more relevant to system apps based on Web 
> technology than Web apps as such; I fear that system-app-motivated complexity 
> is bloating the api.
> - Many of Apple's key target platforms don't even *have* a user-visible 
> filesystem.
> - We'd like to keep Web APIs consistent between our various platforms 
> consistent as much as possible.
> - I think trying to serve the "real filesystem" use cases adds a lot of 
> complexity over the "private storage area" use case.
> - We already have way too many storage APIs. To add another, the use cases 
> would have to be overwhelmingly compelling, and highly impractical to serve 
> by extending any of the other storage APIs (e.g. IndexedDB).
> - In particular, I have heard an explanation of why IndexedDB as it 
> currently exists can't handle all the use cases for file-like storage, but I 
> have heard no explanation of why it can't be extended in that direction.
>
> For these reasons, I think it is unlikely that Safari would ever support 
> Filesystem API in its current form. I could imagine considering a *much* 
> narrower API, scoped only to the use case of private storage areas for Web 
> apps, but only if a compelling case is made that there's no other way to 
> serve that use case.
>
> For what it's worth, even the DeviceStorage API proposal is too complex for 
> my tastes, in its current iteration.
>
> I think keeping the Filseystem API on the REC track in its current form is 
> actively bad, because it leads outside observers to be misled about where the 
> Web platform is going. For example,  sites like  give 
> out "points" for it even though it seems unlikely to advance on the standards 
> track as it stands today.

For what it's worth, I put together a draft for what an API would look
like that has basically the same feature set as the current FileSystem
API, but based on DeviceStorage. It's a much smaller API that the
current FileSystem drafts, but supports things like shallow as well as
deep directory iteration.

https://wiki.mozilla.org/WebAPI/DeviceStorageAPI2

I think that if we at mozilla were to implement a sandboxed
filesystem, it'd be something more like this.

The FileHandle part of the API is definitely more complex to implement
than the FileWriter API, but I'd argue that it's actually easier to
use. For example you can start multiple write operations without
having to wait for each individual write to finish.

I also added an example of what a read-only DeviceStorage would look
like if we wanted something like that for ,
drag'n'drop or a zip-reader. This shouldn't be considered as part of
the remaining draft though since exposing a filesystem in those
scenarios might very well be overkill.

/ Jonas

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

My personal objections (ones that I think are shared by at least some other 
Safari folks):

- It's way too complicated. (As one crude metric, I count 22 interfaces; and 
yes, I know many of those are callback interfaces or sync versions of 
interfaces; it still seems overengineered).
- I see value in the use case of a private sandboxed storage area, as staging 
for uploads, to hold downloaded resources, etc. But:
- It's not totally clear to me why a filesystemlike API is the best way to 
serve these use cases, particularly if in practice it is actually backed by a 
database.
- The API as designed is way too complex for just that use case.
- I'm not keen on exposing portions of the user's filesystem. In particular:
- This seems like a potential security/social-engineering risk.
- This use case seems more relevant to system apps based on Web technology 
than Web apps as such; I fear that system-app-motivated complexity is bloating 
the api.
- Many of Apple's key target platforms don't even *have* a user-visible 
filesystem.
- We'd like to keep Web APIs consistent between our various platforms 
consistent as much as possible.
- I think trying to serve the "real filesystem" use cases adds a lot of 
complexity over the "private storage area" use case.
- We already have way too many storage APIs. To add another, the use cases 
would have to be overwhelmingly compelling, and highly impractical to serve by 
extending any of the other storage APIs (e.g. IndexedDB).
- In particular, I have heard an explanation of why IndexedDB as it 
currently exists can't handle all the use cases for file-like storage, but I 
have heard no explanation of why it can't be extended in that direction.

For these reasons, I think it is unlikely that Safari would ever support 
Filesystem API in its current form. I could imagine considering a *much* 
narrower API, scoped only to the use case of private storage areas for Web 
apps, but only if a compelling case is made that there's no other way to serve 
that use case.

For what it's worth, even the DeviceStorage API proposal is too complex for my 
tastes, in its current iteration.

I think keeping the Filseystem API on the REC track in its current form is 
actively bad, because it leads outside observers to be misled about where the 
Web platform is going. For example,  sites like  give out 
"points" for it even though it seems unlikely to advance on the standards track 
as it stands today.

Regards,
Maciej

On Sep 21, 2012, at 4:32 PM, Eric U  wrote:

> While I don't see any other browsers showing interest in implementing
> the FileSystem API as currently specced, I do see Firefox coming
> around to the belief that a filesystem-style API is a good thing,
> hence their DeviceStorage API.  Rather than scrap the API that we've
> put 2 years of discussion and work into, why not work with us to
> evolve it to something you'd like more?  If you have objections to
> specific attributes of the API, wouldn't it be more efficient to
> change just those things than to start over from scratch?  Or worse,
> to have the Chrome filesystem API, the Firefox filesystem API, etc.?
> 
> If I understand correctly, folks at Mozilla think having a directory
> abstraction is too heavy-weight, and would prefer users to slice and
> dice paths by hand.  OK, that's a small change, and the
> functionality's roughly equivalent.  We could probably even make
> migration fairly easy with a small polyfill.
> 
> Jonas suggests FileHandle to replace FileWriter.  That's clearly not a
> move to greater simplicity, and no polyfill is possible, but it does
> open up the potential for higher perfomance, especially in a
> multi-process browser.  As i said when you proposed it, I'm
> interested, and we'd also like to solve the locking use cases.
> 
> Let's talk about it, rather than throw the baby out with the bathwater.
> 
>   Eric
> 
> On Tue, Sep 18, 2012 at 4:04 AM, Olli Pettay  wrote:
>> Hi all,
>> 
>> 
>> I think we should discuss about moving File API: Directories and System API
>> from Recommendation track to Note. Mainly because the API hasn't been widely
>> accepted nor
>> implemented and also because there are other proposals which handle the same
>> use cases.
>> The problem with keeping the API in recommendation track is that people
>> outside
>> standardization world think that the API is the one which all the browsers
>> will implement and
>> as of now that doesn't seem likely.
>> 
>> 
>> 
>> 
>> -Olli
>> 
> 

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

I like the idea of offering asynchronous listing of files in . But I think Filesystem API is overkill for this use case.

Regards,
Maciej

On Sep 21, 2012, at 3:50 PM, Darin Fisher  wrote:

> No comment on the value of DirectoryEntry for enabling asynchronous listing 
> of files in ?
> 
> -Darin
> 
> On Thu, Sep 20, 2012 at 4:48 PM, Maciej Stachowiak  wrote:
> 
> +1
> 
> I don't see an indication of any major browser but Chrome planning to 
> implement this and expose it to the Web.
> 
>  - Maciej
> 
> On Sep 18, 2012, at 4:04 AM, Olli Pettay  wrote:
> 
> > Hi all,
> >
> >
> > I think we should discuss about moving File API: Directories and System API
> > from Recommendation track to Note. Mainly because the API hasn't been 
> > widely accepted nor
> > implemented and also because there are other proposals which handle the 
> > same use cases.
> > The problem with keeping the API in recommendation track is that people 
> > outside
> > standardization world think that the API is the one which all the browsers 
> > will implement and
> > as of now that doesn't seem likely.
> >
> >
> >
> >
> > -Olli
> >
> 
> 
> 

Re: Moving File API: Directories and System API to Note track?

[Eric U]

While I don't see any other browsers showing interest in implementing
the FileSystem API as currently specced, I do see Firefox coming
around to the belief that a filesystem-style API is a good thing,
hence their DeviceStorage API.  Rather than scrap the API that we've
put 2 years of discussion and work into, why not work with us to
evolve it to something you'd like more?  If you have objections to
specific attributes of the API, wouldn't it be more efficient to
change just those things than to start over from scratch?  Or worse,
to have the Chrome filesystem API, the Firefox filesystem API, etc.?

If I understand correctly, folks at Mozilla think having a directory
abstraction is too heavy-weight, and would prefer users to slice and
dice paths by hand.  OK, that's a small change, and the
functionality's roughly equivalent.  We could probably even make
migration fairly easy with a small polyfill.

Jonas suggests FileHandle to replace FileWriter.  That's clearly not a
move to greater simplicity, and no polyfill is possible, but it does
open up the potential for higher perfomance, especially in a
multi-process browser.  As i said when you proposed it, I'm
interested, and we'd also like to solve the locking use cases.

Let's talk about it, rather than throw the baby out with the bathwater.

Eric

On Tue, Sep 18, 2012 at 4:04 AM, Olli Pettay  wrote:
> Hi all,
>
>
> I think we should discuss about moving File API: Directories and System API
> from Recommendation track to Note. Mainly because the API hasn't been widely
> accepted nor
> implemented and also because there are other proposals which handle the same
> use cases.
> The problem with keeping the API in recommendation track is that people
> outside
> standardization world think that the API is the one which all the browsers
> will implement and
> as of now that doesn't seem likely.
>
>
>
>
> -Olli
>

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

Or.. drag-n-drop receipt of a folder or set of folders.

On Fri, Sep 21, 2012 at 3:50 PM, Darin Fisher  wrote:

> No comment on the value of DirectoryEntry for enabling asynchronous
> listing of files in ?
>
> -Darin
>
>
> On Thu, Sep 20, 2012 at 4:48 PM, Maciej Stachowiak  wrote:
>
>>
>> +1
>>
>> I don't see an indication of any major browser but Chrome planning to
>> implement this and expose it to the Web.
>>
>>  - Maciej
>>
>> On Sep 18, 2012, at 4:04 AM, Olli Pettay  wrote:
>>
>> > Hi all,
>> >
>> >
>> > I think we should discuss about moving File API: Directories and System
>> API
>> > from Recommendation track to Note. Mainly because the API hasn't been
>> widely accepted nor
>> > implemented and also because there are other proposals which handle the
>> same use cases.
>> > The problem with keeping the API in recommendation track is that people
>> outside
>> > standardization world think that the API is the one which all the
>> browsers will implement and
>> > as of now that doesn't seem likely.
>> >
>> >
>> >
>> >
>> > -Olli
>> >
>>
>>
>>
>

Re: Paths exposed in input type=file (was: Re: Moving File API: Directories and System API to Note track?)

[Darin Fisher]

On Fri, Sep 21, 2012 at 1:14 AM, James Graham  wrote:

> On 09/20/2012 11:45 PM, Darin Fisher wrote:
>
>  File path information is already exposed via .
>>
>> "File names may contain partial paths."
>> http://www.whatwg.org/specs/**web-apps/current-work/**
>> multipage/states-of-the-type-**attribute.html#concept-input-**
>> type-file-selected
>>
>
> I couldn't get any actual browser to expose paths in this way. Did I miss
> something? It also seems to be encouraging wilful violation of the FileAPI
> spec which clearly states that File objects don't return path info [1].
>
> Having a may-level conformance criterion that supposes to override a
> must-level criterion in another spec, but no strong compat. requirement,
> doesn't seem like the optimum path to interoperability or sanity of people
> trying to understand and implement the platform. Is there any reason this
> paragraph shouldn't be considered a bug in HTML?
>

I'm not sure.

Note: Chrome exposes relative file path information via
File.webkitRelativePath.  I haven't reviewed the history of this property.

Exposing partial path information like this is useful in conjunction with
 [*].  It enables people to upload directories
of files to a web server.  The web server can re-create the exact directory
structure, including empty directories.

Regards,
-Darin

[*]
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-April/025764.html


>
> [1] 
> http://dev.w3.org/2006/webapi/**FileAPI/#dfn-file
>
>
>

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

No comment on the value of DirectoryEntry for enabling asynchronous listing
of files in ?

-Darin

On Thu, Sep 20, 2012 at 4:48 PM, Maciej Stachowiak  wrote:

>
> +1
>
> I don't see an indication of any major browser but Chrome planning to
> implement this and expose it to the Web.
>
>  - Maciej
>
> On Sep 18, 2012, at 4:04 AM, Olli Pettay  wrote:
>
> > Hi all,
> >
> >
> > I think we should discuss about moving File API: Directories and System
> API
> > from Recommendation track to Note. Mainly because the API hasn't been
> widely accepted nor
> > implemented and also because there are other proposals which handle the
> same use cases.
> > The problem with keeping the API in recommendation track is that people
> outside
> > standardization world think that the API is the one which all the
> browsers will implement and
> > as of now that doesn't seem likely.
> >
> >
> >
> >
> > -Olli
> >
>
>
>

Paths exposed in input type=file (was: Re: Moving File API: Directories and System API to Note track?)

[James Graham]

On 09/20/2012 11:45 PM, Darin Fisher wrote:


File path information is already exposed via .

"File names may contain partial paths."
http://www.whatwg.org/specs/web-apps/current-work/multipage/states-of-the-type-attribute.html#concept-input-type-file-selected


I couldn't get any actual browser to expose paths in this way. Did I 
miss something? It also seems to be encouraging wilful violation of the 
FileAPI spec which clearly states that File objects don't return path 
info [1].


Having a may-level conformance criterion that supposes to override a 
must-level criterion in another spec, but no strong compat. requirement, 
doesn't seem like the optimum path to interoperability or sanity of 
people trying to understand and implement the platform. Is there any 
reason this paragraph shouldn't be considered a bug in HTML?


[1] http://dev.w3.org/2006/webapi/FileAPI/#dfn-file

Re: Moving File API: Directories and System API to Note track?

[Maciej Stachowiak]

+1

I don't see an indication of any major browser but Chrome planning to implement 
this and expose it to the Web.

 - Maciej

On Sep 18, 2012, at 4:04 AM, Olli Pettay  wrote:

> Hi all,
> 
> 
> I think we should discuss about moving File API: Directories and System API
> from Recommendation track to Note. Mainly because the API hasn't been widely 
> accepted nor
> implemented and also because there are other proposals which handle the same 
> use cases.
> The problem with keeping the API in recommendation track is that people 
> outside
> standardization world think that the API is the one which all the browsers 
> will implement and
> as of now that doesn't seem likely.
> 
> 
> 
> 
> -Olli
> 

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

On Wed, Sep 19, 2012 at 11:50 PM, James Graham  wrote:

>
>
> On Wed, 19 Sep 2012, Adam Barth wrote:
>
>  On Wed, Sep 19, 2012 at 1:46 PM, James Graham  wrote:
>>
>>> On Wed, 19 Sep 2012, Edward O'Connor wrote:
>>>
 Olli wrote:

> I think we should discuss about moving File API: Directories and
> System API from Recommendation track to Note.
>

 Sounds good to me.

>>>
>>> Indeed. We are not enthusiastic about implementing an API that has to
>>> traverse directory trees as this has significant technical challenges, or
>>> may expose user's path names, as this has security implications. Also
>>> AIUI
>>> this API is not a good fit for all platforms.
>>>
>>
>> There's nothing in the spec that exposes user paths.  That's just FUD.
>>
>
> I was thinking specifically of the combination of this and Drag and Drop
> and this API. I assumed that at some level one would end up with a bunch on
> Entry objects which seem to expose a path. It then seems that then a user
> who is tricked into dragging their root drive onto a webapp would expose
> all their paths.
>
> It is quite possible that this is a horrible misunderstanding of the spec,
> and if so I apologise. Nevertheless I think it's poor form to immediately
> characterise an error as a deliberate attempt to spread lies.
>
> In any case my central point remains which is that would support this spec
> moving off the Rec. track at this time.
>
>
File path information is already exposed via .

"File names may contain partial paths."
http://www.whatwg.org/specs/web-apps/current-work/multipage/states-of-the-type-attribute.html#concept-input-type-file-selected

-Darin

Re: Moving File API: Directories and System API to Note track?

[Adam Barth]

On Wed, Sep 19, 2012 at 11:50 PM, James Graham  wrote:
> On Wed, 19 Sep 2012, Adam Barth wrote:
>> On Wed, Sep 19, 2012 at 1:46 PM, James Graham  wrote:
>>> On Wed, 19 Sep 2012, Edward O'Connor wrote:
 Olli wrote:
> I think we should discuss about moving File API: Directories and
> System API from Recommendation track to Note.

 Sounds good to me.
>>>
>>> Indeed. We are not enthusiastic about implementing an API that has to
>>> traverse directory trees as this has significant technical challenges, or
>>> may expose user's path names, as this has security implications. Also
>>> AIUI this API is not a good fit for all platforms.
>>
>> There's nothing in the spec that exposes user paths.  That's just FUD.
>
> I was thinking specifically of the combination of this and Drag and Drop and
> this API. I assumed that at some level one would end up with a bunch on
> Entry objects which seem to expose a path. It then seems that then a user
> who is tricked into dragging their root drive onto a webapp would expose all
> their paths.
>
> It is quite possible that this is a horrible misunderstanding of the spec,
> and if so I apologise. Nevertheless I think it's poor form to immediately
> characterise an error as a deliberate attempt to spread lies.

It just has nothing to do with the spec.  It's like complaining that
DOMString might leak user paths because if you use a DOMString with
drag and drop, you might leak user paths.

Adam

Re: Moving File API: Directories and System API to Note track?

[James Graham]

On Wed, 19 Sep 2012, Adam Barth wrote:


On Wed, Sep 19, 2012 at 1:46 PM, James Graham  wrote:

On Wed, 19 Sep 2012, Edward O'Connor wrote:

Olli wrote:

I think we should discuss about moving File API: Directories and
System API from Recommendation track to Note.


Sounds good to me.


Indeed. We are not enthusiastic about implementing an API that has to
traverse directory trees as this has significant technical challenges, or
may expose user's path names, as this has security implications. Also AIUI
this API is not a good fit for all platforms.


There's nothing in the spec that exposes user paths.  That's just FUD.


I was thinking specifically of the combination of this and Drag and Drop 
and this API. I assumed that at some level one would end up with a bunch 
on Entry objects which seem to expose a path. It then seems that then a 
user who is tricked into dragging their root drive onto a webapp would 
expose all their paths.


It is quite possible that this is a horrible misunderstanding of the spec, 
and if so I apologise. Nevertheless I think it's poor form to immediately 
characterise an error as a deliberate attempt to spread lies.


In any case my central point remains which is that would support this spec 
moving off the Rec. track at this time.

Re: Moving File API: Directories and System API to Note track?

[Adam Barth]

On Wed, Sep 19, 2012 at 1:46 PM, James Graham  wrote:
> On Wed, 19 Sep 2012, Edward O'Connor wrote:
>> Olli wrote:
>>> I think we should discuss about moving File API: Directories and
>>> System API from Recommendation track to Note.
>>
>> Sounds good to me.
>
> Indeed. We are not enthusiastic about implementing an API that has to
> traverse directory trees as this has significant technical challenges, or
> may expose user's path names, as this has security implications. Also AIUI
> this API is not a good fit for all platforms.

There's nothing in the spec that exposes user paths.  That's just FUD.

Adam

Re: Moving File API: Directories and System API to Note track?

[James Graham]

On Wed, 19 Sep 2012, Edward O'Connor wrote:


Hi,

Olli wrote:


I think we should discuss about moving File API: Directories and
System API from Recommendation track to Note.


Sounds good to me.



Indeed. We are not enthusiastic about implementing an API that has to 
traverse directory trees as this has significant technical challenges, or 
may expose user's path names, as this has security implications. Also AIUI 
this API is not a good fit for all platforms.

Re: Moving File API: Directories and System API to Note track?

[Edward O'Connor]

Hi,

Olli wrote:

> I think we should discuss about moving File API: Directories and
> System API from Recommendation track to Note.

Sounds good to me.


Ted

Re: Moving File API: Directories and System API to Note track?

[Ian Hickson]

On Wed, 19 Sep 2012, Darin Fisher wrote:
>
> Note: The {File,Directory}Entry types are also separately useful for 
> multi-file input and drag-n-drop applications: 
> http://wiki.whatwg.org/wiki/DragAndDropEntries

...in Chrome. This was the recent topic of:

   http://lists.w3.org/Archives/Public/public-whatwg-archive/2012Sep/0207.html

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

Note: The {File,Directory}Entry types are also separately useful for
multi-file input and drag-n-drop applications:
http://wiki.whatwg.org/wiki/DragAndDropEntries

To summarize, it solves several problems with FileList:

1-  A DirectoryEntry can be lazily resolved by the developer.  With
FileList, the browser has to fully traverse a potentially deep directory
structure before it can provide the application with the FileList.  That
can be very slow.  DirectoryEntry enables applications to provide progress
UI.

2-  FileList is not good at representing empty directories.  Empty
directories can be important for cases where a user wishes to upload a
directory structure to a server.

(This has been implemented in Chrome for a little while now.)

My understand of the sysapps working group was that it was for APIs that
are not generally suitable for use in the conventional web sandbox.  They
are for use in a different ("system apps") sandbox.  What I've described
above is useful in both, and so standardizing in webapps seems to make
sense.

Regards,
-Darin


On Tue, Sep 18, 2012 at 4:11 AM, Robin Berjon  wrote:

> On 18/09/2012 13:04 , Olli Pettay wrote:
>
>> I think we should discuss about moving File API: Directories and
>> System API from Recommendation track to Note. Mainly because the API
>> hasn't been widely accepted nor implemented and also because there
>> are other proposals which handle the same use cases. The problem with
>> keeping the API in recommendation track is that people outside
>> standardization world think that the API is the one which all the
>> browsers will implement and as of now that doesn't seem likely.
>>
>
> I wonder if a good case couldn't be made to move it to the upcoming
> SysApps WG 
> (http://www.w3.org/2012/05/**sysapps-wg-charter.html
> )?
>
> It's very late to sneak anything into the charter, but I reckon it could
> fall under their Media Storage API deliverable.
>
> --
> Robin Berjon - http://berjon.com/ - @robinberjon
>
>

Re: Moving File API: Directories and System API to Note track?

[SULLIVAN, BRYAN L]

What are the other proposals that handle the same use cases?

Thanks,
Bryan Sullivan

On Sep 18, 2012, at 6:07 PM, "Olli Pettay"  wrote:

Hi all,


I think we should discuss about moving File API: Directories and System API
from Recommendation track to Note. Mainly because the API hasn't been widely 
accepted nor
implemented and also because there are other proposals which handle the same 
use cases.
The problem with keeping the API in recommendation track is that people outside
standardization world think that the API is the one which all the browsers will 
implement and
as of now that doesn't seem likely.




-Olli

Re: Moving File API: Directories and System API to Note track?

[SULLIVAN, BRYAN L]

For the use cases of media storage for offline use perhaps (an important use 
case), but there are broader use cases necessary for the growth of Webapps 
which need to manage their own arbitrary storage resources outside the 
browser-managed space. Until this is supported, the web will remain a thin UI 
layer that cannot compete with native applications.

Thanks,
Bryan Sullivan

On Sep 18, 2012, at 6:12 PM, "Robin Berjon"  wrote:

On 18/09/2012 13:04 , Olli Pettay wrote:
> I think we should discuss about moving File API: Directories and
> System API from Recommendation track to Note. Mainly because the API
> hasn't been widely accepted nor implemented and also because there
> are other proposals which handle the same use cases. The problem with
> keeping the API in recommendation track is that people outside
> standardization world think that the API is the one which all the
> browsers will implement and as of now that doesn't seem likely.

I wonder if a good case couldn't be made to move it to the upcoming SysApps WG 
(http://www.w3.org/2012/05/sysapps-wg-charter.html)?

It's very late to sneak anything into the charter, but I reckon it could fall 
under their Media Storage API deliverable.

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Re: Moving File API: Directories and System API to Note track?

[Robin Berjon]

On 18/09/2012 13:04 , Olli Pettay wrote:

I think we should discuss about moving File API: Directories and
System API from Recommendation track to Note. Mainly because the API
hasn't been widely accepted nor implemented and also because there
are other proposals which handle the same use cases. The problem with
keeping the API in recommendation track is that people outside
standardization world think that the API is the one which all the
browsers will implement and as of now that doesn't seem likely.


I wonder if a good case couldn't be made to move it to the upcoming 
SysApps WG (http://www.w3.org/2012/05/sysapps-wg-charter.html)?


It's very late to sneak anything into the charter, but I reckon it could 
fall under their Media Storage API deliverable.


--
Robin Berjon - http://berjon.com/ - @robinberjon

Moving File API: Directories and System API to Note track?

[Olli Pettay]

Hi all,


I think we should discuss about moving File API: Directories and System API
from Recommendation track to Note. Mainly because the API hasn't been widely 
accepted nor
implemented and also because there are other proposals which handle the same 
use cases.
The problem with keeping the API in recommendation track is that people outside
standardization world think that the API is the one which all the browsers will 
implement and
as of now that doesn't seem likely.




-Olli