RE: flush() | was Re: FileSystem API Comments

[Ali Alabbas]

Hi Arun,

We believe that the flush property should be specified when getting the file 
handle as in option 1. One benefit of this is that it will enable the buffer of 
both reads and writes for the same handle. On the other hand, if we specify it 
on every write operation (as in option 2) we could run into inconsistencies 
when invoking the write method with and without the flag.

Based on our interpretation of option 1, it seems as though the flush() 
function would not be available. This is how we perceived its usage would look 
like:

navigator.getFileSystem().then(function(root) {
return root.openWrite("path/to/file.txt", {autoFlushing: true});
}).then(function(fileHandle) {
fileHandle.write(blob);  // data is written to the system cache and is 
flushed to disk without delay
});

Thank you,
Ali


From: Arun Ranganathan [mailto:a...@mozilla.com] 
Sent: Friday, October 31, 2014 11:19 AM
To: Ali Alabbas
Cc: Web Applications Working Group WG
Subject: flush() | was Re: FileSystem API Comments

Greetings Ali!

I've been thinking about the discussion of flush(), and would like to see if I 
can make my previous statement a bit more nuanced. It turns out that flush() 
(in the vein of fsync/sync) is pretty useful, and after discussion with a few 
folks within Mozilla, I realize that it isn't as simple as tacking it on to the 
"write-family" of Promises - as you point out, it is a potentially expensive 
operation.

Something like a flush feature might help the following use cases:

1. Creating a database technology on top of the filesystem technology. This 
might include IndexedDB, but also WebSQL (as a hypothetical example). Most 
transactional operations like this need the ability to do something like flush.

2. Then, there's the use case of compiling C++ codebases to JS. Well-known 
examples of this are games, leveraging asm.js. In this genre of use case, 
sometimes a large database is brought over (e.g. sqlite). It could be memory 
backed, but it is a definite bonus if it could be filesystem backed. Something 
like flush helps make that a possibility.

Now the question is how to do this in a WebAPI, allowing for the power along 
with the mitigations that a web app might need, notably for performance? A few 
ideas below:

On Oct 21, 2014, at 4:36 PM, Ali Alabbas  wrote:


 * flush()
 - This is costly functionality to expose and is likely to be overused by 
callers. It would be beneficial to automatically flush changes to disk by 
allowing the default file write behavior by the OS. For example, on Windows, we 
would leave it up to the filesystem cache to determine the best time to flush 
to disk. This is non-deterministic from the app's point of view, but the only 
time it is a potential problem is when there's a hard power-off. Most apps 
should not be concerned with this; only apps that have very high data 
reliability requirements would need the granular control of flushing to disk. 
In those cases a developer should use IndexedDB. So we should consider 
obscuring this functionality since it's not a common requirement and has a 
performance impact if it's widely used.


I agree with the idea of obscuring the functionality a bit, especially given 
that it might not be necessary for a large class of operations. A few ways to 
do that:

1. Add this to a dictionary option when coining the FileHandleWritable from the 
Directory (e.g. add it to something like the OpeWriteOptions: 
http://w3c.github.io/filesystem-api/Overview.html#widl-Directory-openWrite-Promise-FileHandleWritable--DOMString-File-path-OpenWriteOptions-options).

This way, the developer has the ability to "coin" a "more expensive" promise, 
if that particular set of write operations needs this feature.

2. Add this to the set of options on the FileHandleWritable.

This could be by dictionary, again. Or, it could be a boolean on the 
FileHandleWritable's write(). This latter might not be specific enough. Like 
other implementations, ours is not going to buffer anything, but rely on the 
underlying operating system's buffer for writes and reads.

3. Stick with the idea of a method, like flush(). In this case, we might have 
to caveat the use of this, since the possibility of inexperienced developer 
misuse is high :-) It might help to see if we can determine some boundaries on 
this.

Any feedback on some of these options would be valuable. I am thinking of 1. 
and 2.

- A*

flush() | was Re: FileSystem API Comments

[Arun Ranganathan]

Greetings Ali!

I’ve been thinking about the discussion of flush(), and would like to see if I 
can make my previous statement a bit more nuanced. It turns out that flush() 
(in the vein of fsync/sync) is pretty useful, and after discussion with a few 
folks within Mozilla, I realize that it isn’t as simple as tacking it on to the 
“write-family” of Promises — as you point out, it is a potentially expensive 
operation.

Something like a flush feature might help the following use cases:

1. Creating a database technology on top of the filesystem technology. This 
might include IndexedDB, but also WebSQL (as a hypothetical example). Most 
transactional operations like this need the ability to do something like flush.

2. Then, there’s the use case of compiling C++ codebases to JS. Well-known 
examples of this are games, leveraging asm.js. In this genre of use case, 
sometimes a large database is brought over (e.g. sqlite). It could be memory 
backed, but it is a definite bonus if it could be filesystem backed. Something 
like flush helps make that a possibility.

Now the question is how to do this in a WebAPI, allowing for the power along 
with the mitigations that a web app might need, notably for performance? A few 
ideas below:

On Oct 21, 2014, at 4:36 PM, Ali Alabbas  wrote:

>  * flush()
>  - This is costly functionality to expose and is likely to be overused by 
> callers. It would be beneficial to automatically flush changes to disk by 
> allowing the default file write behavior by the OS. For example, on Windows, 
> we would leave it up to the filesystem cache to determine the best time to 
> flush to disk. This is non-deterministic from the app's point of view, but 
> the only time it is a potential problem is when there's a hard power-off. 
> Most apps should not be concerned with this; only apps that have very high 
> data reliability requirements would need the granular control of flushing to 
> disk. In those cases a developer should use IndexedDB. So we should consider 
> obscuring this functionality since it's not a common requirement and has a 
> performance impact if it's widely used.


I agree with the idea of obscuring the functionality a bit, especially given 
that it might not be necessary for a large class of operations. A few ways to 
do that:

1. Add this to a dictionary option when coining the FileHandleWritable from the 
Directory (e.g. add it to something like the OpeWriteOptions: 
http://w3c.github.io/filesystem-api/Overview.html#widl-Directory-openWrite-Promise-FileHandleWritable--DOMString-File-path-OpenWriteOptions-options).

This way, the developer has the ability to “coin” a “more expensive” promise, 
if that particular set of write operations needs this feature.

2. Add this to the set of options on the FileHandleWritable.

This could be by dictionary, again. Or, it could be a boolean on the 
FileHandleWritable’s write(). This latter might not be specific enough. Like 
other implementations, ours is not going to buffer anything, but rely on the 
underlying operating system’s buffer for writes and reads.

3. Stick with the idea of a method, like flush(). In this case, we might have 
to caveat the use of this, since the possibility of inexperienced developer 
misuse is high :-) It might help to see if we can determine some boundaries on 
this.

Any feedback on some of these options would be valuable. I am thinking of 1. 
and 2.

— A*

Re: FileSystem API Comments

[Arun Ranganathan]

Ali,

First, thanks for your timely comments :) I’m in the process of editing the 
FileSystem API.

Responses inline:

On Oct 21, 2014, at 4:36 PM, Ali Alabbas  wrote:

>  
> 1.1 Use cases (3. Audio/Photo editor with offline access or local cache for 
> speed)
>  
>   * Edited files should be accessible by other client-side applications
>  - Having the sandboxed file system share its contents between all apps 
> would allow apps to tamper with the files of another app. 




Admitedly, these use cases have been borrowed from the “File API: Directories 
and System” specification (which is now a W3C Note), at least for the purpose 
of providing equivalent functionality. In practice, everything you’ve pointed 
out makes it a hard problem to solve. 

The per-origin sandbox model also raises file lock issues on multiple access, 
but they are probably easier to solve, and not as prevalent.

Also, we’re going to forego the “temporary” and “persistent” distinctions that 
are in the draft I think. And while there’s a technical dependency on Quota 
Manager, I don’t think there’s a spec. dependency in terms of API. Of course, 
certain Directory operations may reject a promise with a quota error.


>  
>  
> 3. The Directory Interface
>  
>   * Change events
>  - I would like to revisit the discussion on apps getting notifications 
> of changes to files/directories. 


This is a good point; right now there’s no way to do this. I’m open to 
suggestions. An early scratch pad version of my spec. changes proposed 
Directory as an EventTarget also, but… this won’t work for a variety of reasons.



>  
>   * removeDeep() & move()
>  - Do these support links or junctions? If not, what is the expected 
> behavior?


No; but the entire API doesn’t support these right now.


>  
>   * enumerate()
>  - It would be useful to have pre-filtering support for the following: 
> file/directory, ranges, wildcard search. Currently we would be forced to 
> enumerate the entire set and manually filter out the items we want from the 
> result set.


I completely agree this would be useful, but there’s a problem to solve even 
before we get there! Right now, we say that we’ll fullfill the enumerate 
promise with something called “EventStream” which was initially a Tab Atkins 
proposal, and which would be really useful to get right. It’s underspecified 
right now, but I’m a fan of it ;-)

We’ll have to think about how to return wildcard searches, etc., and how to 
annotate results in the result set.


>  
> 4. The FileHandle Interface
>  
>   * FileHandles
>  - Is this basically going to be the first to get the handle gets to use 
> it and all subsequent calls need to wait for the file handle to become 
> available again? Are there more details about the locking model used here?


Yes; essentially the “first invocation” uses then releases it.

A version of this problem was encountered when specifying FileReader 
(http://dev.w3.org/2006/webapi/FileAPI/#dfn-filereader) which used the internal 
state (but also accessible to the developer) “LOADING” to prevent multiple 
concurrent reads.


>  
>   * Auto-closing of FileHandles
>  - This may cause confusion as it does not match the common developer 
> mental model of a file handle which is “opened” and then available for use 
> until it's “closed”. Perhaps it would be advantageous to have an explicit 
> close function as part of the FileHandle interface? 


There are pros and cons either way. I’d be interested in solving this for the 
lion’s share of use cases. I’m not strongly opinionated on the matter of an 
explicit close function (we have one on Blob, for example), but it seems even 
this has drawbacks.


>   * AbortableProgressPromise
>  - It is not clear how a developer would define the abort callback of an 
> AbortableProgressPromise. It seems that the user agent would be responsible 
> for setting the abort callback since it instantiates and returns the 
> AbortableProgressPromise.


We’re going to not use an AbortableProgressPromise, but we will probably have a 
new beast called CancelablePromise.


>  
>  
> 5. The FileHandleWritable Interface
>  
>   * write() & flush()
>  - It might be useful to have support for “transacted” streams where the 
> caller can write to a copy of the file and then have it atomically replaced: 
> swap the old file with the new one and then delete the old file. 


Agreed.


>  
>   * flush()
>  - This is costly functionality to expose and is likely to be overused by 
> callers. 


Agreed — let’s flush flush().


> 6. FileSystem Configuration Parameters
>  
>   * Dictionary DestinationDict
>  - The DestinationDict seems to exist to facilitate the renaming of a 
> directory in conjunction with a move. However, the same operation is

Re: FileSystem API Comments

[Jonas Sicking]

David is correct.

There certainly are use cases for having Webapps share data. Not just data
in filesystem's, but also structured data stored in something like
localStorage or IndexedDB.

However this filesystem API proposal does not address those use cases.

If someone would like to come up with a proposal for data sharing between
Webapps they should feel free to do so. Such a proposal would need to also
include a description of a security model.

But again, the current proposal does not address that use case.

/ Jonas
On Oct 22, 2014 3:30 AM, "David Rajchenbach-Teller" 
wrote:

> I don't see a contradiction.
> Each *web* app sees only files accessible from its domain (so your two
> apps have distinct "pic.jpeg").
> Each *native* app has access to whatever the operating system says.
>
> Or am I missing something in your message?
>
> Cheers,
>  David
>
> On 22/10/14 12:23, Jonathan Bond-Caron wrote:
> > That contradicts:
> > - Edited files should be accessible by other client-side applications
> >
> > The api should allow for editing a 'shared folder' which multiple
> applications / web apps can access.
> > That implies a sort of locking/unlocking api:
> >
> > e.g.
> > photo editor
> > fs = api.getFileSystem({shareName: "photos"}).then((dir) => {
> dir.openWrite("pic.jpeg") });
> >
> > super photo viewer
> > fs = api.getFileSystem({shareName: "photos"}).then((dir) => {
> dir.openRead("pic.jpeg") });
> >
> > What happens with the pic.jpeg?
> >
> >
>
>
> --
> David Rajchenbach-Teller, PhD
>  Performance Team, Mozilla
>
>

Re: FileSystem API Comments

[chaals]

22.10.2014, 12:32, "David Rajchenbach-Teller" :
> I don't see a contradiction.
> Each *web* app sees only files accessible from its domain (so your two
> apps have distinct "pic.jpeg").
> Each *native* app has access to whatever the operating system says.

There are a lot of use cases for sharing data with apps of *different* origins, 
although there is of course a more complex security story than when everything 
goes into a potentially opaque sandbox. (And to make the basic security story 
work it makes sense to have some level of opacity in the sandbox).

The lack of a mechanism to do so is a huge difference with native - I have 
directories in my filesystem that are autosynched to things online, but are 
also visible.

The idea behind web intents/activites/etc generalises obviously to remove the 
distinction between web and native - I should be able to use a web-based image 
manipulation tool on stuff in my filesystem. Or several.

At the moment that can be done in a somewhat hacky way by uploading files, 
manipulating them, then asking the user to save them back. But whereas I have 
mail clients that store each email message on the filesystem, so I can import 
stuff into a different program myself instead of having to go through a service 
provider, that doesn't work for web-based email systems even when those are 
designed to be functional offline.

etc etc.

cheers

Chaals

> Or am I missing something in your message?
>
> Cheers,
>  David
>
> On 22/10/14 12:23, Jonathan Bond-Caron wrote:
>>  That contradicts:
>>  - Edited files should be accessible by other client-side applications
>>
>>  The api should allow for editing a 'shared folder' which multiple 
>> applications / web apps can access.
>>  That implies a sort of locking/unlocking api:
>>
>>  e.g.
>>  photo editor
>>  fs = api.getFileSystem({shareName: "photos"}).then((dir) => { 
>> dir.openWrite("pic.jpeg") });
>>
>>  super photo viewer
>>  fs = api.getFileSystem({shareName: "photos"}).then((dir) => { 
>> dir.openRead("pic.jpeg") });
>>
>>  What happens with the pic.jpeg?
>
> --
> David Rajchenbach-Teller, PhD
>  Performance Team, Mozilla

--
Charles McCathie Nevile - web standards - CTO Office, Yandex
cha...@yandex-team.ru - - - Find more at http://yandex.com

Re: FileSystem API Comments

[David Rajchenbach-Teller]

I don't see a contradiction.
Each *web* app sees only files accessible from its domain (so your two
apps have distinct "pic.jpeg").
Each *native* app has access to whatever the operating system says.

Or am I missing something in your message?

Cheers,
 David

On 22/10/14 12:23, Jonathan Bond-Caron wrote:
> That contradicts:
> - Edited files should be accessible by other client-side applications
> 
> The api should allow for editing a 'shared folder' which multiple 
> applications / web apps can access.
> That implies a sort of locking/unlocking api:
> 
> e.g.
> photo editor
> fs = api.getFileSystem({shareName: "photos"}).then((dir) => { 
> dir.openWrite("pic.jpeg") });
> 
> super photo viewer
> fs = api.getFileSystem({shareName: "photos"}).then((dir) => { 
> dir.openRead("pic.jpeg") });
> 
> What happens with the pic.jpeg?
> 
> 


-- 
David Rajchenbach-Teller, PhD
 Performance Team, Mozilla



signature.asc
Description: OpenPGP digital signature

RE: FileSystem API Comments

[Jonathan Bond-Caron]

On Tue Oct 21 09:36 PM, Jonas Sicking wrote:
> > 1.1 Use cases (3. Audio/Photo editor with offline access or local
> > cache for
> > speed)
> >
> >   * Edited files should be accessible by other client-side
> > applications
> >
> >  - Having the sandboxed file system share its contents between all
> > apps would allow apps to tamper with the files of another app. This
> > could result in corrupted files and perhaps an invalid state for some
> > apps that expect certain contents to exist in a file. This makes us
> > wonder: should we warn users about files that are being opened and
> > written to? 
> 
> Each origin has a separate sandboxed filesystem. There is no way for websites
> to read each other's filesystems. This is no different from IndexedDB or
> localStorage. This also means that we have the same prompting behavior, the
> same Quota Management dependency and the same security model as
> IndexedDB and localStorage.
> 

That contradicts:
- Edited files should be accessible by other client-side applications

The api should allow for editing a 'shared folder' which multiple applications 
/ web apps can access.
That implies a sort of locking/unlocking api:

e.g.
photo editor
fs = api.getFileSystem({shareName: "photos"}).then((dir) => { 
dir.openWrite("pic.jpeg") });

super photo viewer
fs = api.getFileSystem({shareName: "photos"}).then((dir) => { 
dir.openRead("pic.jpeg") });

What happens with the pic.jpeg?

Re: FileSystem API Comments

[Jonas Sicking]

On Tue, Oct 21, 2014 at 1:36 PM, Ali Alabbas  wrote:
> Hello,
>
> I'm with the IE Platform team at Microsoft. We have a few comments on the
> latest editor's draft of the newly proposed FileSystem API [1].

Hi Ali! Thanks for looking at the spec. I'm glad you've been able to
understand the API as well as you have, despite it's "lacking"
descriptions of the API. To say the least :)

> 1.1 Use cases (3. Audio/Photo editor with offline access or local cache for
> speed)
>
>   * Edited files should be accessible by other client-side applications
>
>  - Having the sandboxed file system share its contents between all apps
> would allow apps to tamper with the files of another app. This could result
> in corrupted files and perhaps an invalid state for some apps that expect
> certain contents to exist in a file. This makes us wonder: should we warn
> users about files that are being opened and written to? If an app is just
> doing a read, can it open a file or directory without the user's permission,
> or could this pose a possible issue as well? Also, is the Quota Management
> API going to be a dependency? It's unclear what we would do with regards to
> requesting permission to access files. Will this spec be responsible for
> defining what questions/permission inquiries are presented and when they are
> presented to the user? For example, what happens when one file is locked for
> use by a different application? Is the user notified and given the option to
> open a read-only copy of that file?

Each origin has a separate sandboxed filesystem. There is no way for
websites to read each other's filesystems. This is no different from
IndexedDB or localStorage. This also means that we have the same
prompting behavior, the same Quota Management dependency and the same
security model as IndexedDB and localStorage.

Different browsers have used various different policies for IndexedDB
and localStorage in the past. Firefox originally prompted as soon as
you used IndexedDB, but we currently instead prompt once you store
more than 50MB. Our plan is to very soon remove all prompts. I think
Safari on iOS prompts once you reach a certain limit of stored data,
in WebSQL, not sure if this applies to localStorage and IndexedDB as
well. Chrome, Opera and IE has never prompted I think.

My hope is that we can agree on some policies for this stuff and
encode that in a Quota Management specification. Once we do that it
should hopefully apply across all storage APIs. I.e. hopefully what
will matter is how much data a website stores, not what API it uses to
store that data.

The "Edited files should be accessible by other client-side
applications" refers to the ability for non-web applications to access
these files. I.e. if the user has iTunes or WinAmp installed then it
might be good for those native applications to be able to scan files
stored by websites to try to find music files stored there. This
technically is no different in IndexedDB. I.e. it'd be cool if
iTunes/WinAmp could find music files stored as Blobs in IndexedDB.

I'm personally happy to remove this requirement as I don't think it
affects the API at all.

> 3. The Directory Interface
>   * Change events
>
>  - I would like to revisit the discussion on apps getting notifications
> of changes to files/directories. There are many scenarios where an
> application would want to react to renames/moves of a file/directory. There
> would also be value in being notified of a change to a directory's
> structure. If an app has a file browser that allows a user to select files
> and/or directories and another app makes changes to the sandboxed
> filesystem, then it would be expected that the first app should be notified
> and would be able to refresh its directory tree. Otherwise it would require
> the user to somehow force a refresh which would not be a good user
> experience since the user would expect the file browser to update on its
> own.

I agree. The lack of change notifications has been a problem for
IndexedDB. The second most common feedback that we get is that change
notifications are needed. Only exceeded by "Why no Promises you
jerkfaces!"

>   * removeDeep() & move()
>
>  - Do these support links or junctions? If not, what is the expected
> behavior?

Links and junctions aren't supported in the API at all right now. I.e.
there's no way to create a link/junction, and so the problem of what
happens if you delete/move them does not appear.

>   * enumerate()
>
>  - It would be useful to have pre-filtering support for the following:
> file/directory, ranges, wildcard search. Currently we would be forced to
> enumerate the entire set and manually filter out the items we want from the
> result set.

Re: FileSystem API Comments

[Arthur Barstow]

On 10/21/14 4:36 PM, Ali Alabbas wrote:


Hello,

I'm with the IE Platform team at Microsoft. We have a few comments on 
the latest editor's draft of the newly proposed FileSystem API [1].





I believe [1] is Arun's <http://w3c.github.io/filesystem-api/Overview.html>.

1.1 Use cases (3. Audio/Photo editor with offline access or local 
cache for speed)


* Edited files should be accessible by other client-side applications

- Having the sandboxed file system share its contents between all apps 
would allow apps to tamper with the files of another app. This could 
result in corrupted files and perhaps an invalid state for some apps 
that expect certain contents to exist in a file. This makes us wonder: 
should we warn users about files that are being opened and written to? 
If an app is just doing a read, can it open a file or directory 
without the user's permission, or could this pose a possible issue as 
well? Also, is the Quota Management API going to be a dependency? It's 
unclear what we would do with regards to requesting permission to 
access files. Will this spec be responsible for defining what 
questions/permission inquiries are presented and when they are 
presented to the user? For example, what happens when one file is 
locked for use by a different application? Is the user notified and 
given the option to open a read-only copy of that file?


3. The Directory Interface

* Change events

- I would like to revisit the discussion on apps getting notifications 
of changes to files/directories. There are many scenarios where an 
application would want to react to renames/moves of a file/directory. 
There would also be value in being notified of a change to a 
directory's structure. If an app has a file browser that allows a user 
to select files and/or directories and another app makes changes to 
the sandboxed filesystem, then it would be expected that the first app 
should be notified and would be able to refresh its directory tree. 
Otherwise it would require the user to somehow force a refresh which 
would not be a good user experience since the user would expect the 
file browser to update on its own.


* removeDeep() & move()

- Do these support links or junctions? If not, what is the expected 
behavior?


* enumerate()

- It would be useful to have pre-filtering support for the following: 
file/directory, ranges, wildcard search. Currently we would be forced 
to enumerate the entire set and manually filter out the items we want 
from the result set.


- Callers often know exactly whether or not they want to enumerate 
files or folders. For example, an image upload service may only be 
interested in the files present in a directory rather than all of its 
directories. Perhaps it would be useful to have enumerateFiles() and 
enumerateDirectories() for this purpose? Or we could have another 
argument for enumerate() that is an enum ("directory", "file").


- Supporting optimized pagination of large directories. We could have 
arguments for a starting index and length we would be able to specify 
a range of items to retrieve within the result set.


- Supporting the wildcard character to pre-filter a list of 
files/directories (e.g. *.jpg).


4. The FileHandle Interface

* FileHandles

- Is this basically going to be the first to get the handle gets to 
use it and all subsequent calls need to wait for the file handle to 
become available again? Are there more details about the locking model 
used here?


* Auto-closing of FileHandles

- This may cause confusion as it does not match the common developer 
mental model of a file handle which is “opened” and then available for 
use until it's “closed”. Perhaps it would be advantageous to have an 
explicit close function as part of the FileHandle interface? With the 
current behavior there can be overhead with the unintended closure of 
the FileHandle that would require a developer to continuously 
open/close a FileHandle. The currently defined behavior assumes that a 
developer is done with all their file manipulations when they have 
completed a promise chain. However, a developer may want to keep the 
FileHandle open to be used elsewhere at some other point in time that 
is not related to the current promise chain. An example of the 
usefulness of having an explicit close function is if you were to 
implement a word processor and wanted to lock down the file that it 
currently has open for the period of its editing. This way you are 
free to continue operating on that file for the duration that it is 
open, protecting the file from other processes, and not having to 
undergo the costly setup and teardown of a file handle.


* AbortableProgressPromise

- It is not clear how a developer would define the abort callback of 
an AbortableProgressPromise. It seems that the user agent would be 
responsible for setting the abort callback since it instantiates and 
returns the Abortab

FileSystem API Comments

[Ali Alabbas]

Hello,

I'm with the IE Platform team at Microsoft. We have a few comments on the 
latest editor's draft of the newly proposed FileSystem API [1].

1.1 Use cases (3. Audio/Photo editor with offline access or local cache for 
speed)

  * Edited files should be accessible by other client-side applications
 - Having the sandboxed file system share its contents between all apps 
would allow apps to tamper with the files of another app. This could result in 
corrupted files and perhaps an invalid state for some apps that expect certain 
contents to exist in a file. This makes us wonder: should we warn users about 
files that are being opened and written to? If an app is just doing a read, can 
it open a file or directory without the user's permission, or could this pose a 
possible issue as well? Also, is the Quota Management API going to be a 
dependency? It's unclear what we would do with regards to requesting permission 
to access files. Will this spec be responsible for defining what 
questions/permission inquiries are presented and when they are presented to the 
user? For example, what happens when one file is locked for use by a different 
application? Is the user notified and given the option to open a read-only copy 
of that file?


3. The Directory Interface

  * Change events
 - I would like to revisit the discussion on apps getting notifications of 
changes to files/directories. There are many scenarios where an application 
would want to react to renames/moves of a file/directory. There would also be 
value in being notified of a change to a directory's structure. If an app has a 
file browser that allows a user to select files and/or directories and another 
app makes changes to the sandboxed filesystem, then it would be expected that 
the first app should be notified and would be able to refresh its directory 
tree. Otherwise it would require the user to somehow force a refresh which 
would not be a good user experience since the user would expect the file 
browser to update on its own.

  * removeDeep() & move()
 - Do these support links or junctions? If not, what is the expected 
behavior?

  * enumerate()
 - It would be useful to have pre-filtering support for the following: 
file/directory, ranges, wildcard search. Currently we would be forced to 
enumerate the entire set and manually filter out the items we want from the 
result set.
 - Callers often know exactly whether or not they want to enumerate 
files or folders. For example, an image upload service may only be interested 
in the files present in a directory rather than all of its directories. Perhaps 
it would be useful to have enumerateFiles() and enumerateDirectories() for this 
purpose? Or we could have another argument for enumerate() that is an enum 
("directory", "file").
 - Supporting optimized pagination of large directories. We could have 
arguments for a starting index and length we would be able to specify a range 
of items to retrieve within the result set.
 - Supporting the wildcard character to pre-filter a list of 
files/directories (e.g. *.jpg).

4. The FileHandle Interface

  * FileHandles
 - Is this basically going to be the first to get the handle gets to use it 
and all subsequent calls need to wait for the file handle to become available 
again? Are there more details about the locking model used here?

  * Auto-closing of FileHandles
 - This may cause confusion as it does not match the common developer 
mental model of a file handle which is "opened" and then available for use 
until it's "closed". Perhaps it would be advantageous to have an explicit close 
function as part of the FileHandle interface? With the current behavior there 
can be overhead with the unintended closure of the FileHandle that would 
require a developer to continuously open/close a FileHandle. The currently 
defined behavior assumes that a developer is done with all their file 
manipulations when they have completed a promise chain. However, a developer 
may want to keep the FileHandle open to be used elsewhere at some other point 
in time that is not related to the current promise chain. An example of the 
usefulness of having an explicit close function is if you were to implement a 
word processor and wanted to lock down the file that it currently has open for 
the period of its editing. This way you are free to continue operating on that 
file for the duration that it is open, protecting the file from other 
processes, and not having to undergo the costly setup and teardown of a file 
handle.

  * AbortableProgressPromise
 - It is not clear how a developer would define the abort callback of an 
AbortableProgressPromise. It seems that the user agent would be responsible for 
setting the abort callback since it instantiates and returns the 
AbortableProgressPromise.


5. The FileHandleWritable Interface

  * w

FileSystem API behavior of seek function

[Kyle Graehl]

I am writing to give my feedback on the behavior of the FileWriter seek
function.

From
http://www.w3.org/TR/file-writer-api/#widl-FileWriter-seek-void-long-long-offset,
we see that for FileWriter.seek, "If position > length then set position to
length." (where length is the length of the file)

It would be really nice if seeking past the end of the file would cause
zeros to be written in the byte range [length, position-1], just how POSIX
does it. I have an application that needs to write past the file boundary
and so emulates POSIX behavior in JavaScript user space land, but it is
quite slow and inefficient compared to having it be done by the underlying
system call.

Not matching the behavior of POSIX seek seems like a poor design decision
to me. Was the current behavior chosen for simplicity of implementation?

If the behavior of FileWriter seek were changed to match the POSIX
behavior, how would applications be able to predict which version of the
FileWriter seek implementation were being used?

Re: Polished FileSystem API proposal

[Brian Stell]

There are multiple interesting ideas being discussed

1. Mapping files to persistent URLs.
2. Sharing persistent URLs between different origins.
3. Using the ServiceWorker [1] to redirect URL requests (and possibly
manage it's own cache / files)
4. De-duping file copies using a Git like scheme.

1. Mapping files to persistent URLs.
==
There are some things that could be considered 'ok' to be slow: images,
video startup, list of unread emails. There are some things that are very
time sensitive such as the initial page layout. This means that things like
the CSS required for layout and the fonts required for layout need to be
there before the body begins laying out; ie: they need to be available in
the  section. If they are not then the page will irritatingly flash
as the parts arrive. Unless I'm missing something, this means that objects
retrieved by promises/callbacks are not fast enough.

For these "needed for layout" objects, persistent URLs are very attractive
as they can be retrieved quickly and in the  section. These
persistent URLs can be implemented in a filesystem or  IndexedDB or other
mechanism. A ServiceWorker could remap the URL but the data would still
need to be available locally (eg, on disk).


2. Sharing persistent URLs between different origins.
==
Right now, any interesting object that is used by different origins (even
from the same organization) must be downloaded and stored once per origin.
Imagine if Linux required glib to be separately stored for every
executable. This is how the web works today.

Shareable persistent URLs would allow for a single copy to be download and
shared across origins. Like shared libraries the user of the shared object
has to trust the providing origin and only the providing origin should be
able to write the data.


3. Using the ServiceWorker to redirect URL requests (and possibly manage
it's own cache / files)
==
The ServiceWorker provides a way for an web page to redirect URLs. This is
a very attractive feature for applications that are offline (or have an
unreliable connection). The redirected URL could be to a completely
different URL or to data managed by the ServiceWorker itself; eg, the
ServiceWorker could use the FileSystem API to store data and redirect URLs
to that data. Hopefully, this redirection will be fast; eg, fast enough for
'needed for layout' objects.

Each ServiceWorker is origin specific: "they are not shared across domains,
and they are completely isolated from the browser's HTTP cache" [2]. I take
this to imply that the ServiceWorker has no ability to provide persistent
URLs to other origins.


4. De-duping file copies using a Git like scheme.
==
My sense is everyone likes the idea of avoiding storing redundant data and
Git's use of the SHA1 message digest as the filename is 'good enough'. This
is a low security risk mechanism that is good for storage efficiency. The
most benefit occurs when the storage mechanism (eg, FileSystem, IndexedDB)
applies this across origins. Like sharing across origins it gets the
benefit of avoiding duplicates but it does not address the multiple
downloads issue. Multiple downloads are probably okay for smallish files
but could be an issue for larger files such as 20Mbyte Chinese fonts, large
Javascript libraries, etc. My wild guess is that because this is a 'good
thing to do' but not 'a critical thing to do', its odds of getting
implemented are poor.


Brian Stell


Notes:
[1] https://github.com/slightlyoff/ServiceWorker
[2] https://github.com/slightlyoff/ServiceWorker/blob/master/caching.md






On Wed, Nov 6, 2013 at 8:28 AM, pira...@gmail.com  wrote:

> > That's very interesting and useful, but I don't think it fits the same
> use
> > case I was talking about.  I want the ability to create some object that
> > exports an URL that I can put in an iframe.  Then all requests from that
> > iframe for resources will dynamically call my javascript code.  I could
> > implement the same logic that a server-side application does, but from
> local
> > code in my browser.
> >
> That's just the purpose of ServiceWorker :-) Only that from your
> message, I suspect you are asking about having the same functionality
> but only on the current session or maybe also only when the page is
> open, deleting it on reload. I don't know of anything like to this,
> the most similar ones would be FirefoxOS Browser API or Chrome
> FileSystem API, but nothing as powerful as ServiceWorker, sorry :-(
> They are talking about implementing the Fetch specification, maybe you
> would write them about allowing to be used someway the Service

Re: Polished FileSystem API proposal

[pira...@gmail.com]

> That's very interesting and useful, but I don't think it fits the same use
> case I was talking about.  I want the ability to create some object that
> exports an URL that I can put in an iframe.  Then all requests from that
> iframe for resources will dynamically call my javascript code.  I could
> implement the same logic that a server-side application does, but from local
> code in my browser.
>
That's just the purpose of ServiceWorker :-) Only that from your
message, I suspect you are asking about having the same functionality
but only on the current session or maybe also only when the page is
open, deleting it on reload. I don't know of anything like to this,
the most similar ones would be FirefoxOS Browser API or Chrome
FileSystem API, but nothing as powerful as ServiceWorker, sorry :-(
They are talking about implementing the Fetch specification, maybe you
would write them about allowing to be used someway the ServiceWorker
functionality on a per-session basis, I find legitimate your
proposition...



-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Polished FileSystem API proposal

[Tim Caswell]

On Wed, Nov 6, 2013 at 8:46 AM, pira...@gmail.com  wrote:

> > Yes.  I completely agree.  Personally in my projects, I wish there was to
> > create custom filesystems backed by JavaScript code that are exported to
> > hierarchical URLs.  It would work somewhat like server-side web
> development
> > where a js function handles the request for a file.  This is basically
> what
> > fuse does for Linux.
> >
> > I would love to be able to create a custom union filesystem that has
> layers.
> > One would be in-memory and another would be backed by a local bit
> database.
> > It would all be implemented in pure user space js.
> >
> Maybe you would be interested on ServiceWorkers
>
> https://github.com/slightlyoff/ServiceWorker
>
> They are currently being implemented on Firefox and Chrome.


That's very interesting and useful, but I don't think it fits the same use
case I was talking about.  I want the ability to create some object that
exports an URL that I can put in an iframe.  Then all requests from that
iframe for resources will dynamically call my javascript code.  I could
implement the same logic that a server-side application does, but from
local code in my browser.

My particular use case is I'm developing an IDE to teach programming and
would love to generate dynamic file systems that are then rendered in
iframes or new windows.

Having a better offline store than appcache is indeed useful, but I believe
it's orthogonal to what I'm looking for here.

Like Jonas said, "Use hierarchical filesystem: URLs" is exactly what I'm
looking for and currently I can only do this in chrome with their html5 fs
api.  But instead of creating a temporary filesystem and writing files, I'd
much rather if my code was called for every file request and I could
dynamically generate the files on demand.



>
> --
> "Si quieres viajar alrededor del mundo y ser invitado a hablar en un
> monton de sitios diferentes, simplemente escribe un sistema operativo
> Unix."
> – Linus Tordvals, creador del sistema operativo Linux
>

Re: Polished FileSystem API proposal

[pira...@gmail.com]

> Yes.  I completely agree.  Personally in my projects, I wish there was to
> create custom filesystems backed by JavaScript code that are exported to
> hierarchical URLs.  It would work somewhat like server-side web development
> where a js function handles the request for a file.  This is basically what
> fuse does for Linux.
>
> I would love to be able to create a custom union filesystem that has layers.
> One would be in-memory and another would be backed by a local bit database.
> It would all be implemented in pure user space js.
>
Maybe you would be interested on ServiceWorkers

https://github.com/slightlyoff/ServiceWorker

They are currently being implemented on Firefox and Chrome.


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Polished FileSystem API proposal

[Tim Caswell]

On Nov 6, 2013 3:50 AM, "Jonas Sicking"  wrote:
>
> On Tue, Nov 5, 2013 at 11:45 AM, Tim Caswell  wrote:
> > If the backend implementation used something like git's data store then
> > duplicate data would automatically be stored only once without any
security
> > implications.  The keys are the literal sha1 of the values.  If two
websites
> > had the same file tree containing the same files, it would be the same
tree
> > object in the storage.  But only sites who have a reference to the hash
> > would have access to it.
> >
> > Also I like the level of fs support that git's filesystem has.  There
are
> > trees, files, executable files, and symlinks. (there are also gitlinks
used
> > for submodules, but let's ignore those for now)
>
> Sounds like IndexedDB is a better fit than a filesystem for this use case.

I'm not saying that I want a filesystem so I can implement git.  I'm saying
that git is a great way to implement a filesystem and that git's abilities
are a great baseline for web filesystems.

>
> Note that the use case for the filesystem API isn't "storing files",
> IDB is perfectly capable of doing that. The use case for the
> filesystem API is to satisfy people that want a true filesystem with
> directories etc so that they can:
>
> * Sync to a server-side file system. For example when doing web
> development and want to deploy a website.
> * Use hierarchical filesystem: URLs.
> * Support in-place editing of large files.
> * Because filesystems are familiar.
>

Yes.  I completely agree.  Personally in my projects, I wish there was to
create custom filesystems backed by JavaScript code that are exported to
hierarchical URLs.  It would work somewhat like server-side web development
where a js function handles the request for a file.  This is basically what
fuse does for Linux.

I would love to be able to create a custom union filesystem that has
layers.  One would be in-memory and another would be backed by a local bit
database.  It would all be implemented in pure user space js.

> A simple key-value storage, where the values happen to be files,
> doesn't need a filesystem API.

Speaking of, I currently have a git implementation that uses IndexedDB as
the data store.  I don't call IDB simple by any means.  I don't need
transactions or indexes.  The leveldb interface is much closer. To what I
need to implement git.

>
> / Jonas

So in summary:

- The filesystem interface could export symlinks, files, trees, and
executable bit.   (Basically feature parity with git trees)
- using a git style backing store (the content addressable part) is a great
way to store data from multiple domains without duplicating storage or
introducing security issues around sharing data.
- I really want a fuse-like way to create virtual file systems where I
implement the back end using user space code.
- I want a key/value system that's simple like local storage, but async and
supports binary values.

Sorry for conflating so many topics in one email.  They are all related and
I'm new to this list.  I'm not sure of the proper procedure for creating
new topics.

- Tim Caswell

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Tue, Nov 5, 2013 at 11:45 AM, Tim Caswell  wrote:
> If the backend implementation used something like git's data store then
> duplicate data would automatically be stored only once without any security
> implications.  The keys are the literal sha1 of the values.  If two websites
> had the same file tree containing the same files, it would be the same tree
> object in the storage.  But only sites who have a reference to the hash
> would have access to it.
>
> Also I like the level of fs support that git's filesystem has.  There are
> trees, files, executable files, and symlinks. (there are also gitlinks used
> for submodules, but let's ignore those for now)

Sounds like IndexedDB is a better fit than a filesystem for this use case.

Note that the use case for the filesystem API isn't "storing files",
IDB is perfectly capable of doing that. The use case for the
filesystem API is to satisfy people that want a true filesystem with
directories etc so that they can:

* Sync to a server-side file system. For example when doing web
development and want to deploy a website.
* Use hierarchical filesystem: URLs.
* Support in-place editing of large files.
* Because filesystems are familiar.

A simple key-value storage, where the values happen to be files,
doesn't need a filesystem API.

/ Jonas

Re: Polished FileSystem API proposal

[Tim Caswell]

If the backend implementation used something like git's data store then
duplicate data would automatically be stored only once without any security
implications.  The keys are the literal sha1 of the values.  If two
websites had the same file tree containing the same files, it would be the
same tree object in the storage.  But only sites who have a reference to
the hash would have access to it.

Also I like the level of fs support that git's filesystem has.  There are
trees, files, executable files, and symlinks. (there are also gitlinks used
for submodules, but let's ignore those for now)


On Tue, Nov 5, 2013 at 12:19 PM, Anne van Kesteren  wrote:

> On Thu, Oct 31, 2013 at 2:12 AM, Brian Stell  wrote:
> > There could be *dozens* of copies of exactly the same a Javascript
> library,
> > shared CSS, or web font in the FileSystem.
>
> Check out the cache part of
> https://github.com/slightlyoff/ServiceWorker/ Combined with a smart
> implementation that will do exactly what you want. And avoid all the
> issues of an actual cross-origin file system API.
>
>
> --
> http://annevankesteren.nl/
>
>

Re: Polished FileSystem API proposal

[Brian Stell]

I like Git's model :-)

This would de-dup the file storage but won't it require downloading it for
every domain (when the data is not lingering in HTTP cache)?




On Tue, Nov 5, 2013 at 11:45 AM, Tim Caswell  wrote:

> If the backend implementation used something like git's data store then
> duplicate data would automatically be stored only once without any security
> implications.  The keys are the literal sha1 of the values.  If two
> websites had the same file tree containing the same files, it would be the
> same tree object in the storage.  But only sites who have a reference to
> the hash would have access to it.
>
> Also I like the level of fs support that git's filesystem has.  There are
> trees, files, executable files, and symlinks. (there are also gitlinks used
> for submodules, but let's ignore those for now)
>
>
> On Tue, Nov 5, 2013 at 12:19 PM, Anne van Kesteren wrote:
>
>> On Thu, Oct 31, 2013 at 2:12 AM, Brian Stell  wrote:
>> > There could be *dozens* of copies of exactly the same a Javascript
>> library,
>> > shared CSS, or web font in the FileSystem.
>>
>> Check out the cache part of
>> https://github.com/slightlyoff/ServiceWorker/ Combined with a smart
>> implementation that will do exactly what you want. And avoid all the
>> issues of an actual cross-origin file system API.
>>
>>
>> --
>> http://annevankesteren.nl/
>>
>>
>

Re: Polished FileSystem API proposal

[Anne van Kesteren]

On Thu, Oct 31, 2013 at 2:12 AM, Brian Stell  wrote:
> There could be *dozens* of copies of exactly the same a Javascript library,
> shared CSS, or web font in the FileSystem.

Check out the cache part of
https://github.com/slightlyoff/ServiceWorker/ Combined with a smart
implementation that will do exactly what you want. And avoid all the
issues of an actual cross-origin file system API.


-- 
http://annevankesteren.nl/

Re: Polished FileSystem API proposal

[Brian Stell]

On Wed, Oct 30, 2013 at 7:19 PM, pira...@gmail.com wrote:

> What you are asking for could be fixed with redirects, that it's the
> HTTP equivalent of filesystems symbolic links :-)
>

Is your suggestion that Google consolidate all its domains into one?

These are widely separated servers (internet wise) with widely separated
teams with widely separated schedules.

In addition to different teams/schedules separate domains are important to
internet load balancing:
   * Search, www.google.com, gets around 2 trillion searches per day [1].
   * During Christmas YouTube got 1.6 million requests per second [2]
   * GMail has nearly 1/2 a billion active users per month [3]

Do you really want that redirected to one domain?

Brian

Notes
[1] http://www.statisticbrain.com/google-searches/
[2] http://www.youtube.com/watch?v=Jq-VMZK1KGk
[3] http://venturebeat.com/2012/06/28/gmail-hotmail-yahoo-email-users/



>
> 2013/10/31 Brian Stell :
> > In "Request for feedback: Filesystem API" [1] it says "This filesystem
> would
> > be origin-specific".
> >
> > This post discusses limited readonly sharing of filesystem resources
> between
> > origins.
> >
> > To improve web site / application performance I'm interested in caching
> > static [2] resources (eg, Javascript libraries, common CSS, fonts) in the
> > filesystem and accessing them thru persistent URLs.
> >
> > So, what is the issue?
> >
> > I'd like to avoid duplication. Consider the following sites: they are all
> > from a single organization but have different specific origins;
> >* https://mail.google.com/
> >* https://plus.google.com/
> >* https://sites.google.com/
> >* ...
> >
> > At google there are *dozens* of these origins [3]. Even within a single
> page
> > there are iframes from different origins. (There are other things that
> lead
> > to different origins but for this post I'm ignoring them [4].)
> >
> > There could be *dozens* of copies of exactly the same a Javascript
> library,
> > shared CSS, or web font in the FileSystem.
> >
> > What I'm suggesting is:
> >* a filesystem's persistent URLs by default be read/write only for the
> > same origin
> >* the origin be able to allow other origins to access its files
> > (readonly) by persistent URL
> >
> > I'm not asking-for nor suggesting API file access but others may express
> > opinions on this.
> >
> > Brian Stell
> >
> >
> > PS: Did I somehow miss info on same-origin in the spec [7]?
> >
> > Notes:
> > [1]
> >
> http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html
> > [2] I'm also assuming immutability would be handled similar to
> gstatic.com
> > [6] where different versions of a file have a different path/filename;
> eg,
> >* V8: http://gstatic.com/fonts/roboto/v8/2UX7WLTfW3W8TclTUvlFyQ.woff
> >* V9: http://gstatic.com/fonts/roboto/v9/2UX7WLTfW3W8TclTUvlFyQ.woff
> >
> > [3] Here are some of Google's origins:
> > https://accounts.google.com
> > https://blogsearch.google.com
> > https://books.google.com
> > https://chrome.google.com
> > https://cloud.google.com
> > https://code.google.com
> > https://csi.gstatic.com
> > https://developers.google.com
> > https://docs.google.com
> > https://drive.google.com
> > https://earth.google.com
> > https://fonts.googleapis.com
> > https://groups.google.com
> > https://mail.google.com
> > https://maps.google.com
> > https://news.google.com
> > https://www.panoramio.com
> > https://picasa.google.com
> > https://picasaweb.google.com
> > https://play.google.com
> > https://productforums.google.com
> > https://plus.google.com/
> > https://research.google.com
> > https://support.google.com
> > https://sites.google.com
> > https://ssl.gstatic.com
> > https://translate.google.com
> > https://tables.googlelabs.com
> > https://talkgadget.google.com
> > https://themes.googleusercontent.com/
> > https://www.blogger.com
> > https://www.google.com
> > https://www.gstatic.com
> > https://www.orcut.com
> > https://www.youtube.com
> >
> > My guess is that there are more.
> >
> > I believe the XXX.blogspot.com origins belong to Google but I'm not an
> > authority on this.
> >
> > [4] These are also different top level domains:
> >* https://www.google.nl
> >* https://www.google.co.jp
> >
> > Wikipedia lists about 200 of these [5] but since users tend to stick to
> one
> > I'm ignoring them for this posting.
> >
> > I'm also ignoring http vs https (eg, http://www.google.com) and
> with/without
> > leading www (eg, https://google.com) since they redirect.
> >
> > [5] http://en.wikipedia.org/wiki/List_of_Google_domains
> > [6] http://wiki.answers.com/Q/What_is_gstatic
> > [7] http://w3c.github.io/filesystem-api/Overview.html
>
>
>
> --
> "Si quieres viajar alrededor del mundo y ser invitado a hablar en un
> monton de sitios diferentes, simplemente escribe un sistema operativo
> Unix."
> – Linus Tordvals, creador del sistema operativo Linux
>

Re: Polished FileSystem API proposal

[pira...@gmail.com]

What you are asking for could be fixed with redirects, that it's the
HTTP equivalent of filesystems symbolic links :-)

2013/10/31 Brian Stell :
> In "Request for feedback: Filesystem API" [1] it says "This filesystem would
> be origin-specific".
>
> This post discusses limited readonly sharing of filesystem resources between
> origins.
>
> To improve web site / application performance I'm interested in caching
> static [2] resources (eg, Javascript libraries, common CSS, fonts) in the
> filesystem and accessing them thru persistent URLs.
>
> So, what is the issue?
>
> I'd like to avoid duplication. Consider the following sites: they are all
> from a single organization but have different specific origins;
>* https://mail.google.com/
>* https://plus.google.com/
>* https://sites.google.com/
>* ...
>
> At google there are *dozens* of these origins [3]. Even within a single page
> there are iframes from different origins. (There are other things that lead
> to different origins but for this post I'm ignoring them [4].)
>
> There could be *dozens* of copies of exactly the same a Javascript library,
> shared CSS, or web font in the FileSystem.
>
> What I'm suggesting is:
>* a filesystem's persistent URLs by default be read/write only for the
> same origin
>* the origin be able to allow other origins to access its files
> (readonly) by persistent URL
>
> I'm not asking-for nor suggesting API file access but others may express
> opinions on this.
>
> Brian Stell
>
>
> PS: Did I somehow miss info on same-origin in the spec [7]?
>
> Notes:
> [1]
> http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html
> [2] I'm also assuming immutability would be handled similar to gstatic.com
> [6] where different versions of a file have a different path/filename; eg,
>* V8: http://gstatic.com/fonts/roboto/v8/2UX7WLTfW3W8TclTUvlFyQ.woff
>* V9: http://gstatic.com/fonts/roboto/v9/2UX7WLTfW3W8TclTUvlFyQ.woff
>
> [3] Here are some of Google's origins:
> https://accounts.google.com
> https://blogsearch.google.com
> https://books.google.com
> https://chrome.google.com
> https://cloud.google.com
> https://code.google.com
> https://csi.gstatic.com
> https://developers.google.com
> https://docs.google.com
> https://drive.google.com
> https://earth.google.com
> https://fonts.googleapis.com
> https://groups.google.com
> https://mail.google.com
> https://maps.google.com
> https://news.google.com
> https://www.panoramio.com
> https://picasa.google.com
> https://picasaweb.google.com
> https://play.google.com
> https://productforums.google.com
> https://plus.google.com/
> https://research.google.com
> https://support.google.com
> https://sites.google.com
> https://ssl.gstatic.com
> https://translate.google.com
> https://tables.googlelabs.com
> https://talkgadget.google.com
> https://themes.googleusercontent.com/
> https://www.blogger.com
> https://www.google.com
> https://www.gstatic.com
> https://www.orcut.com
> https://www.youtube.com
>
> My guess is that there are more.
>
> I believe the XXX.blogspot.com origins belong to Google but I'm not an
> authority on this.
>
> [4] These are also different top level domains:
>* https://www.google.nl
>* https://www.google.co.jp
>
> Wikipedia lists about 200 of these [5] but since users tend to stick to one
> I'm ignoring them for this posting.
>
> I'm also ignoring http vs https (eg, http://www.google.com) and with/without
> leading www (eg, https://google.com) since they redirect.
>
> [5] http://en.wikipedia.org/wiki/List_of_Google_domains
> [6] http://wiki.answers.com/Q/What_is_gstatic
> [7] http://w3c.github.io/filesystem-api/Overview.html



-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Polished FileSystem API proposal

[Brian Stell]

In "Request for feedback: Filesystem API" [1] it says "This filesystem
would be origin-specific".

This post discusses limited readonly sharing of filesystem resources
between origins.

To improve web site / application performance I'm interested in caching
static [2] resources (eg, Javascript libraries, common CSS, fonts) in the
filesystem and accessing them thru persistent URLs.

So, what is the issue?

I'd like to avoid duplication. Consider the following sites: they are all
from a single organization but have different specific origins;
   * https://mail.google.com/
   * https://plus.google.com/
   * https://sites.google.com/
   * ...

At google there are *dozens* of these origins [3]. Even within a single
page there are iframes from different origins. (There are other things that
lead to different origins but for this post I'm ignoring them [4].)

There could be *dozens* of copies of exactly the same a Javascript library,
shared CSS, or web font in the FileSystem.

What I'm suggesting is:
   * a filesystem's persistent URLs by default be read/write only for the
same origin
   * the origin be able to allow other origins to access its files
(readonly) by persistent URL

I'm not asking-for nor suggesting API file access but others may express
opinions on this.

Brian Stell


PS: Did I somehow miss info on same-origin in the spec [7]?

Notes:
[1]
http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html
[2] I'm also assuming immutability would be handled similar to gstatic.com [6]
where different versions of a file have a different path/filename; eg,
   * V8: http://gstatic.com/fonts/roboto/v8/2UX7WLTfW3W8TclTUvlFyQ.woff
   * V9: http://gstatic.com/fonts/roboto/v9/2UX7WLTfW3W8TclTUvlFyQ.woff

[3] Here are some of Google's origins:
https://accounts.google.com
https://blogsearch.google.com
https://books.google.com
https://chrome.google.com
https://cloud.google.com
https://code.google.com
https://csi.gstatic.com
https://developers.google.com
https://docs.google.com
https://drive.google.com
https://earth.google.com
https://fonts.googleapis.com
https://groups.google.com
https://mail.google.com
https://maps.google.com
https://news.google.com
https://www.panoramio.com
https://picasa.google.com
https://picasaweb.google.com
https://play.google.com
https://productforums.google.com
https://plus.google.com/
https://research.google.com
https://support.google.com
https://sites.google.com
https://ssl.gstatic.com
https://translate.google.com
https://tables.googlelabs.com
https://talkgadget.google.com
https://themes.googleusercontent.com/
https://www.blogger.com
https://www.google.com
https://www.gstatic.com
https://www.orcut.com
https://www.youtube.com

My guess is that there are more.

I believe the XXX.blogspot.com origins belong to Google but I'm not an
authority on this.

[4] These are also different top level domains:
   * https://www.google.nl
   * https://www.google.co.jp

Wikipedia lists about 200 of these [5] but since users tend to stick to one
I'm ignoring them for this posting.

I'm also ignoring http vs https (eg, http://www.google.com) and
with/without leading www (eg, https://google.com) since they redirect.

[5] http://en.wikipedia.org/wiki/List_of_Google_domains
[6] http://wiki.answers.com/Q/What_is_gstatic
[7] http://w3c.github.io/filesystem-api/Overview.html

Re: Polished FileSystem API proposal

[pira...@gmail.com]

On most unix OSes, symbolic links are build using plain text files
with just the path where they point inside and no more data, and later
the OS identify them as a link instead a text file just by some
special file flags, no more. On Windows, the direct access (.lnk) has
a somewhat similar functionality (files that has the location of other
files), and there was some discussions on the Wine and some Fat FS
mail lists to use them to mimic real symbolic links.

Hard links are more dificult to implement, since they are real link to
a file, and usually this needs to have a counter of how many
references are pointing to them so it doesn't get accidentally
deleted, and this need support on the filesystem itself, while as I
told you before, symbolic links can be mimic in several ways on a
higher layer.

2013/10/30 Brian Stell :
> Good points! I was thinking of the logical functioning and hadn't considered
> the implementation. My understanding is that the UA will map from the
> filename to an actual file using some kind of database. My assumption was
> the logical idea of a link would happen in that layer.
>
>
> On Wed, Oct 30, 2013 at 1:14 AM, pira...@gmail.com 
> wrote:
>>
>> +1 to symbolic links, they have almost the same functionality that hard
>> links and are more secure and flexible (they are usually just plain text
>> files...).
>>
>> El 30/10/2013 01:42, "Brendan Eich"  escribió:
>>
>>> Hard links are peculiar to Unix filesystems. Not interoperable across all
>>> OSes. Symbolic links, OTOH...
>>>
>>> /be
>>>
 Brian Stell 
 October 29, 2013 4:53 PM
 I meant

eg, V1/dir1/file1, V2/dir1/file1.
>>>
>>>
>



-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Polished FileSystem API proposal

[Brian Stell]

Good points! I was thinking of the logical functioning and hadn't
considered the implementation. My understanding is that the UA will map
from the filename to an actual file using some kind of database. My
assumption was the logical idea of a link would happen in that layer.


On Wed, Oct 30, 2013 at 1:14 AM, pira...@gmail.com wrote:

> +1 to symbolic links, they have almost the same functionality that hard
> links and are more secure and flexible (they are usually just plain text
> files...).
> El 30/10/2013 01:42, "Brendan Eich"  escribió:
>
> Hard links are peculiar to Unix filesystems. Not interoperable across all
>> OSes. Symbolic links, OTOH...
>>
>> /be
>>
>>  Brian Stell 
>>> October 29, 2013 4:53 PM
>>> I meant
>>>
>>>eg, V1/dir1/file1, V2/dir1/file1.
>>>
>>
>>

Re: Polished FileSystem API proposal

[pira...@gmail.com]

+1 to symbolic links, they have almost the same functionality that hard
links and are more secure and flexible (they are usually just plain text
files...).
El 30/10/2013 01:42, "Brendan Eich"  escribió:

> Hard links are peculiar to Unix filesystems. Not interoperable across all
> OSes. Symbolic links, OTOH...
>
> /be
>
>  Brian Stell 
>> October 29, 2013 4:53 PM
>> I meant
>>
>>eg, V1/dir1/file1, V2/dir1/file1.
>>
>
>

Re: Polished FileSystem API proposal

[Brendan Eich]

Hard links are peculiar to Unix filesystems. Not interoperable across 
all OSes. Symbolic links, OTOH...


/be


Brian Stell 
October 29, 2013 4:53 PM
I meant

   eg, V1/dir1/file1, V2/dir1/file1.

Re: Polished FileSystem API proposal

[Brian Stell]

I meant

   eg, V1/dir1/file1, V2/dir1/file1.

Re: Polished FileSystem API proposal

[Brian Stell]

Hi Jonas,

I notice that one of the common Linux file APIs, link[1], is not in you
API. I don't see this as a first pass requirement but I certainly expect
that applications will want to be able to have trees that represent a
particular version of their immutable files; eg, V1/dir1/file1,
V1/dir1/file1. It would possible to copy the unchanged files but that would
double the storage size.

Could you kindly share your thoughts on having a link API?

Thanks,

Brian Stell


Notes:
[1] http://linux.die.net/man/2/link

Re: FileSystem API

[Jonas Sicking]

resending to the right address.

On Mon, Aug 26, 2013 at 4:58 AM, Arthur Barstow  wrote:
> Hi Eric, Arun, Jonas, All,
>
>
>
> On 8/19/13 7:44 PM, ext Eric U wrote:
>>
>> OK, I just finished making my way through the public-script-coord
>> thread [I'm not on that list, but someone pointed me to it].  I have
>> no official objections to you editing a spec based on Jonas's
>> proposal, but I do have a couple of questions:
>>
>> 1) Why is this on public-script-coord instead of public-webapps?
>
>
> I think Jonas' reply on this question covers the rationale. (I [incorrectly]
> assumed everyone in WebApps is subscribed to p-s-c so I'll try to clarify
> our lists' usage when people join the group.)
>
>
>> 2) Is any vendor other than Mozilla actually interested in this
>> proposal?  When it was brought up on public-webapps, and at the
>> WebApps F2F, it dropped with a resounding thud.
>>
>> Given the standardization failure of the Chrome FileSystem API, this
>> could be a massive waste of time.  Or it could just be a way for
>> Mozilla to document its filesystem API, since we've already got
>> documentation of the Chrome API, but then you don't need to drag
>> public-script-coord into that.
>
>
> These are good questions and points. I don't feel real strongly here re our
> next steps other than I think we should try to get consensus on a high-level
> plan to help set expectations accordingly.
>
> It seems we have a few options, some are not necessarily mutually exclusive
> ...
>
> 1. Leave Eric's specs in WD state

I'm totally ok with this one. Though see more below.

> 2. Move Eric's specs to LC->CR (feature set should probably be
> restricted/limited to what is already implemented in Chrome); block in CR
> until there are two or more implementations

I don't support this one. Last time we talked about this API at a face
to face all browser vendors except google explicitly said that they
were not interested at implementing this API.

I don't know why we didn't at the time move this spec to a Note? Does
anyone remember? Does the minutes say?

In any case I think moving this API forward would be sending the wrong
signals to the outside world.

> 3. Move Eric's specs to WG Notes and stop work (as was done with Web SQL
> Database)

This is my preferred action. For the same reason that we moved Web SQL
to a Note. It's very hard for the outside world to know what our
current thinking is as a WG other than through the status of the
various drafts. We can always move this API back from Note to WD if
other vendors express interest in implementing. But I think at this
point this API does not have enough support to remain as a WD as far
as I can tell. Do we have any other active drafts with only support
from a single vendor.

> 4. Merge the two proposals

Like Arun, I don't think this is possible.

> 5. Formally start work on Mozilla proposal knowing there is some overlap
> with Eric's specs
>
> 6. Other options?

Another option is to not start the new API as an official draft until
there is public support from two vendors. But if we're going to do
that I'd like to insist that we move the google API to Note status as
to not signal to the outside world that we still think the google API
is going to be the one that is the future of the web and has more
support than the mozilla proposal.

> It appears the current proposal/plan is #1 plus #5 which is effectively the
> `Darwinism` plan. Is this correct?

I'm ok with this option too.

/ Jonas

Re: FileSystem API

[Arun Ranganathan]

On Aug 26, 2013, at 8:01 AM, Arthur Barstow wrote:

> Hi Eric, Arun, Jonas, All,
> 
> 
>> 2) Is any vendor other than Mozilla actually interested in this
>> proposal?  When it was brought up on public-webapps, and at the
>> WebApps F2F, it dropped with a resounding thud.
>> 
>> Given the standardization failure of the Chrome FileSystem API, this
>> could be a massive waste of time.  Or it could just be a way for
>> Mozilla to document its filesystem API, since we've already got
>> documentation of the Chrome API, but then you don't need to drag
>> public-script-coord into that.
> 
> These are good questions and points. I don't feel real strongly here re our 
> next steps other than I think we should try to get consensus on a high-level 
> plan to help set expectations accordingly.
> 
> It seems we have a few options, some are not necessarily mutually exclusive 
> ...
> 
> 1. Leave Eric's specs in WD state
> 
> 2. Move Eric's specs to LC->CR (feature set should probably be 
> restricted/limited to what is already implemented in Chrome); block in CR 
> until there are two or more implementations
> 
> 3. Move Eric's specs to WG Notes and stop work (as was done with Web SQL 
> Database)
> 
> 4. Merge the two proposals
> 
> 5. Formally start work on Mozilla proposal knowing there is some overlap with 
> Eric's specs


I don't actually think it's possible to merge the two proposals.  I'm in favor 
of option 5.

-- A*

Re: FileSystem API

[Arthur Barstow]

Hi Eric, Arun, Jonas, All,


On 8/19/13 7:44 PM, ext Eric U wrote:

OK, I just finished making my way through the public-script-coord
thread [I'm not on that list, but someone pointed me to it].  I have
no official objections to you editing a spec based on Jonas's
proposal, but I do have a couple of questions:

1) Why is this on public-script-coord instead of public-webapps?


I think Jonas' reply on this question covers the rationale. (I 
[incorrectly] assumed everyone in WebApps is subscribed to p-s-c so I'll 
try to clarify our lists' usage when people join the group.)



2) Is any vendor other than Mozilla actually interested in this
proposal?  When it was brought up on public-webapps, and at the
WebApps F2F, it dropped with a resounding thud.

Given the standardization failure of the Chrome FileSystem API, this
could be a massive waste of time.  Or it could just be a way for
Mozilla to document its filesystem API, since we've already got
documentation of the Chrome API, but then you don't need to drag
public-script-coord into that.


These are good questions and points. I don't feel real strongly here re 
our next steps other than I think we should try to get consensus on a 
high-level plan to help set expectations accordingly.


It seems we have a few options, some are not necessarily mutually 
exclusive ...


1. Leave Eric's specs in WD state

2. Move Eric's specs to LC->CR (feature set should probably be 
restricted/limited to what is already implemented in Chrome); block in 
CR until there are two or more implementations


3. Move Eric's specs to WG Notes and stop work (as was done with Web SQL 
Database)


4. Merge the two proposals

5. Formally start work on Mozilla proposal knowing there is some overlap 
with Eric's specs


6. Other options?

It appears the current proposal/plan is #1 plus #5 which is effectively 
the `Darwinism` plan. Is this correct?


-ArtB

Re: FileSystem API

[Jonas Sicking]

On Mon, Aug 19, 2013 at 4:44 PM, Eric U  wrote:
> OK, I just finished making my way through the public-script-coord
> thread [I'm not on that list, but someone pointed me to it].  I have
> no official objections to you editing a spec based on Jonas's
> proposal, but I do have a couple of questions:
>
> 1) Why is this on public-script-coord instead of public-webapps?

I started on public-webapps, but once discussion died out here I also
wanted to get input from TC39. Generally we're trying to improve
coordination between W3C and TC39 so public-script-coord was set up as
an attempt to help that. I'm hoping that we'll run all major API
designs past them going forward. Not just file-related things, but any
API that we design here and in other WGs.

There were so far many more changes based on discussions on this list
than discussions on public-script-coord.

However public-webapps is still the main list for discussing this API.
If any major changes are coming out of the discussion with TC39 the
intent is to bring those proposed changes back here.

> 2) Is any vendor other than Mozilla actually interested in this
> proposal?  When it was brought up on public-webapps, and at the
> WebApps F2F, it dropped with a resounding thud.

I'll let others speak up, but yes, I've heard positive signals from
other vendors. If we hadn't I wouldn't have let the discussions go as
far as they have.

/ Jonas

> On Mon, Aug 19, 2013 at 3:11 PM, Arun Ranganathan  wrote:
>> Greetings Eric and WG,
>>
>> The Chair and I were discussing setting up repositories for the 
>> specifications discussed here 
>> (http://lists.w3.org/Archives/Public/public-webapps/2013JulSep/0307.html), 
>> notably the FileSystem API and File API v2.  Before creating a repository to 
>> edit the FileSystem API, we thought we'd check with you about the first 
>> proposal, which Chrome implements, and get the Google perspective.
>>
>> You've edited the first FileSystem API proposal, which currently lives here 
>> (http://www.w3.org/TR/file-system-api/).  Can I create a repository and edit 
>> the other proposal for FileSystem API, which currently exists as an email 
>> thread 
>> (http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html)
>>  ?
>>
>> Just checking to see if there are any objections or concerns that would stop 
>> a draft or future WG activity.  Of course, technical nits should be heard as 
>> well, and can proceed concurrently with a draft :)
>>
>> -- A*
>

Re: FileSystem API

[Eric U]

OK, I just finished making my way through the public-script-coord
thread [I'm not on that list, but someone pointed me to it].  I have
no official objections to you editing a spec based on Jonas's
proposal, but I do have a couple of questions:

1) Why is this on public-script-coord instead of public-webapps?
2) Is any vendor other than Mozilla actually interested in this
proposal?  When it was brought up on public-webapps, and at the
WebApps F2F, it dropped with a resounding thud.

Given the standardization failure of the Chrome FileSystem API, this
could be a massive waste of time.  Or it could just be a way for
Mozilla to document its filesystem API, since we've already got
documentation of the Chrome API, but then you don't need to drag
public-script-coord into that.

I may have a few small bits of feedback on the color of the bikeshed,
but mostly I'm going to stay out of it, lest I accidentally give the
impression that we're going to implement it.  As I stated at the F2F,
we'll be the last ones to do it, but if 2 major browser vendors ship
it first, we'll certainly consider it.

On Mon, Aug 19, 2013 at 3:11 PM, Arun Ranganathan  wrote:
> Greetings Eric and WG,
>
> The Chair and I were discussing setting up repositories for the 
> specifications discussed here 
> (http://lists.w3.org/Archives/Public/public-webapps/2013JulSep/0307.html), 
> notably the FileSystem API and File API v2.  Before creating a repository to 
> edit the FileSystem API, we thought we'd check with you about the first 
> proposal, which Chrome implements, and get the Google perspective.
>
> You've edited the first FileSystem API proposal, which currently lives here 
> (http://www.w3.org/TR/file-system-api/).  Can I create a repository and edit 
> the other proposal for FileSystem API, which currently exists as an email 
> thread 
> (http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html)
>  ?
>
> Just checking to see if there are any objections or concerns that would stop 
> a draft or future WG activity.  Of course, technical nits should be heard as 
> well, and can proceed concurrently with a draft :)
>
> -- A*

FileSystem API

[Arun Ranganathan]

Greetings Eric and WG,

The Chair and I were discussing setting up repositories for the specifications 
discussed here 
(http://lists.w3.org/Archives/Public/public-webapps/2013JulSep/0307.html), 
notably the FileSystem API and File API v2.  Before creating a repository to 
edit the FileSystem API, we thought we'd check with you about the first 
proposal, which Chrome implements, and get the Google perspective.

You've edited the first FileSystem API proposal, which currently lives here 
(http://www.w3.org/TR/file-system-api/).  Can I create a repository and edit 
the other proposal for FileSystem API, which currently exists as an email 
thread 
(http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html) ?

Just checking to see if there are any objections or concerns that would stop a 
draft or future WG activity.  Of course, technical nits should be heard as 
well, and can proceed concurrently with a draft :)

-- A*

Re: Polished FileSystem API proposal

[pira...@gmail.com]

>> Hell yes!!! +1000 to add an interface to inotify :-D
> I'm concerned that you're taking this API too far. What's next -
> mmaping a file to a byte array?

It would be cool... :-P

Ok, let's get serious. Maybe inotify would be too high level API and I
agree it would be moved to a later version, but I find it a really
useful feature. in a per-domain sandboxed filesystem maybe it would
not be so important (except if it's accessed from several pages on the
same domain), but if it's a sandboxed mountpoint defined by the user
on a folder on the computer filesystem and he has direct access to it,
if he modify the contect of this sandboxed folder/mountpoint the
webapp wouldn't know about that modifications, and for my personal use
case I'm really interested on this feature (in fact, I proposed it
some time ago, not remember if on this list or on Chrome...).


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Mon, Jul 22, 2013 at 6:56 PM, Kinuko Yasuda  wrote:
> On Tue, Jul 23, 2013 at 4:48 AM, Jonas Sicking  wrote:
>>
>> On Mon, Jul 22, 2013 at 11:18 AM, Jan Varga  wrote:
>> > On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking  wrote:
>> >>
>> >> Hi All,
>> >>
>> >> Yesterday a few of us at mozilla went through the FileSystem API
>> >> proposal we previously sent [1] and tightened it up.
>> >>
>> >
>> > It was also pointed out that we should address multi-file locking too.
>> > One of the options is to make openRead() and openWrite() take a
>> > sequence.
>> >
>> > interface Directory {
>> >   Promise openRead((DOMString or File) file);
>> >   Promise openWrite((DOMString or File) file,
>> > OpenWriteOptions options);
>> >   Promise> openRead(sequence<(DOMString or File)>
>> > files);
>> >   Promise> openWrite(sequence<(DOMString or
>> > File)> files, OpenWriteOptions options);
>> > }
>> >
>> > So, this works with the current proposal, Jonas has a more complex
>> > solution
>> > that probably requires bigger changes in the proposed file system API.
>> > I'll let Jonas to describe it.
>>
>> First of all, I'm not sure that multi-file-locking is something that
>> we need to solve in this API at all. At least not yet. The google API
>> doesn't support locking of any type (as far as I can tell), so I think
>> starting with locking on a single-file basis is a better and simpler
>> place to start.
>>
>> The main concern I have with this approach is that it creates a very
>> specific API which solves a pretty narrow problem of copying data
>> between two files. It doesn't support copying data between a server
>> and a file, or between indexedDB and a file.
>>
>> I can think of two more generic solutions that will solve these problems:
>>
>> A) Adding something like inotify.
>> I.e. add the ability to get notifications about changes to part of a
>> filesystem. This way a page could create a lock-file when it wants to
>> prevent others from accessing a particular part of the filesystem.
>>
>> If the lockfile already exists, it could register to be notified when
>> the lockfile is removed. Once the file is removed it would create the
>> file and start accessing the files in whatever way it wants.
>>
>> This is how multi-process access to application data is often handled
>> in filesystems today.
>>
>> Adding something like inotify would also enable use cases like having
>> a worker synchronize a filesystem to a server. Other parts of the
>> application could simply access the filesystem directly and do
>> whatever modifications it wants. Those notifications will
>> automatically be noticed and synchronized to the server by the worker.
>>
>> B) Add a generic "cross-window lock" mechanism.
>>
>> Basically an API for asynchronously requesting a lock. Once the lock
>> becomes available the caller would be notified. When the caller is
>> done using the lock, he/she calls a function to explicitly release the
>> lock, thus enabling other callers to get notified that they are now
>> holding the lock.
>>
>> A lock is never automatically released by the platform on some timeout
>> or similar. The only time the lock is forcefully released is when the
>> user closes the page that created the lock.
>>
>> It is the application's responsibility to determine when it is
>> appropriate to grab the lock. I.e. there is no platform connection
>> between a lock and the resources that it protects. This way a lock
>> could represent anything from the whole filesystem, a couple of IDB
>> database and some server resources, to just parts of a file.
>>
>> Darin Fisher has proposed something similar in the past as I recall
>> it. Though the details might not exactly match the above.
>>
>>
>> Pros of A
>> - Solves more use cases than simply locking-related ones. We've
>> discussed adding similar observer mechanisms to IDB because the same
>> non-locking-related usecases have come up there.
>> - Enables building something like solution B on top of it.
>>
>> Cons of A
>> - More complicated to use correctly. For example it's important to
>> register for the notification before checking if the lock file is
>> already created. Otherwise there's a risk that the lock file is
>> removed between the time when the page checks if its there and

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Mon, Jul 22, 2013 at 3:40 PM, Janusz Majnert  wrote:
> 2013/7/22 pira...@gmail.com :
>>> Adding something like inotify would also enable use cases like having
>>> a worker synchronize a filesystem to a server. Other parts of the
>>> application could simply access the filesystem directly and do
>>> whatever modifications it wants. Those notifications will
>>> automatically be noticed and synchronized to the server by the worker.
>>>
>> Hell yes!!! +1000 to add an interface to inotify :-D
> I'm concerned that you're taking this API too far. What's next -
> mmaping a file to a byte array?
> It was supposed to be simple, sandboxed API for storing data in
> file-like structures - let's maybe get this thing working first?

FWIW in many other storage APIs, there has been an established need to
enable observing modifications of that storage area.

localStorage had this feature from the beginning.
In IndexedDB discussions between Google and Mozilla determined that
both had received requests for this. Not sure if Microsoft has
received similar feedback.

Even storage OS-level storage APIs, i.e. filesystems, generally have
this functionality.

So I think it makes a lot of sense to add it to this filesystem too.

That said. I'm happy to punt on this for now. As long as we are ok
with punting on support for multi-file locking.

/ Jonas

Re: Polished FileSystem API proposal

[Kinuko Yasuda]

On Tue, Jul 23, 2013 at 4:48 AM, Jonas Sicking  wrote:

> On Mon, Jul 22, 2013 at 11:18 AM, Jan Varga  wrote:
> > On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking  wrote:
> >>
> >> Hi All,
> >>
> >> Yesterday a few of us at mozilla went through the FileSystem API
> >> proposal we previously sent [1] and tightened it up.
> >>
> >
> > It was also pointed out that we should address multi-file locking too.
> > One of the options is to make openRead() and openWrite() take a sequence.
> >
> > interface Directory {
> >   Promise openRead((DOMString or File) file);
> >   Promise openWrite((DOMString or File) file,
> > OpenWriteOptions options);
> >   Promise> openRead(sequence<(DOMString or File)>
> > files);
> >   Promise> openWrite(sequence<(DOMString or
> > File)> files, OpenWriteOptions options);
> > }
> >
> > So, this works with the current proposal, Jonas has a more complex
> solution
> > that probably requires bigger changes in the proposed file system API.
> > I'll let Jonas to describe it.
>
> First of all, I'm not sure that multi-file-locking is something that
> we need to solve in this API at all. At least not yet. The google API
> doesn't support locking of any type (as far as I can tell), so I think
> starting with locking on a single-file basis is a better and simpler
> place to start.
>
> The main concern I have with this approach is that it creates a very
> specific API which solves a pretty narrow problem of copying data
> between two files. It doesn't support copying data between a server
> and a file, or between indexedDB and a file.
>
> I can think of two more generic solutions that will solve these problems:
>
> A) Adding something like inotify.
> I.e. add the ability to get notifications about changes to part of a
> filesystem. This way a page could create a lock-file when it wants to
> prevent others from accessing a particular part of the filesystem.
>
> If the lockfile already exists, it could register to be notified when
> the lockfile is removed. Once the file is removed it would create the
> file and start accessing the files in whatever way it wants.
>
> This is how multi-process access to application data is often handled
> in filesystems today.
>
> Adding something like inotify would also enable use cases like having
> a worker synchronize a filesystem to a server. Other parts of the
> application could simply access the filesystem directly and do
> whatever modifications it wants. Those notifications will
> automatically be noticed and synchronized to the server by the worker.
>
> B) Add a generic "cross-window lock" mechanism.
>
> Basically an API for asynchronously requesting a lock. Once the lock
> becomes available the caller would be notified. When the caller is
> done using the lock, he/she calls a function to explicitly release the
> lock, thus enabling other callers to get notified that they are now
> holding the lock.
>
> A lock is never automatically released by the platform on some timeout
> or similar. The only time the lock is forcefully released is when the
> user closes the page that created the lock.
>
> It is the application's responsibility to determine when it is
> appropriate to grab the lock. I.e. there is no platform connection
> between a lock and the resources that it protects. This way a lock
> could represent anything from the whole filesystem, a couple of IDB
> database and some server resources, to just parts of a file.
>
> Darin Fisher has proposed something similar in the past as I recall
> it. Though the details might not exactly match the above.
>
>
> Pros of A
> - Solves more use cases than simply locking-related ones. We've
> discussed adding similar observer mechanisms to IDB because the same
> non-locking-related usecases have come up there.
> - Enables building something like solution B on top of it.
>
> Cons of A
> - More complicated to use correctly. For example it's important to
> register for the notification before checking if the lock file is
> already created. Otherwise there's a risk that the lock file is
> removed between the time when the page checks if its there and the
> notification is registered.
>
> Pros of B
> - Easier to use than A
>
> Cons of B
> - Doesn't help with use cases other than locking. I.e. to enable one
> window to notice when another window modified some data, you would
> either have to use polling, or create custom signaling mechanisms and
> make sure to use those whenever something in the filesystem is
> modified.


I vote for B, it's more generic, si

Re: Polished FileSystem API proposal

[Janusz Majnert]

2013/7/22 pira...@gmail.com :
>> Adding something like inotify would also enable use cases like having
>> a worker synchronize a filesystem to a server. Other parts of the
>> application could simply access the filesystem directly and do
>> whatever modifications it wants. Those notifications will
>> automatically be noticed and synchronized to the server by the worker.
>>
> Hell yes!!! +1000 to add an interface to inotify :-D
I'm concerned that you're taking this API too far. What's next -
mmaping a file to a byte array?
It was supposed to be simple, sandboxed API for storing data in
file-like structures - let's maybe get this thing working first?

/Janusz Majnert

Re: Polished FileSystem API proposal

[pira...@gmail.com]

> Adding something like inotify would also enable use cases like having
> a worker synchronize a filesystem to a server. Other parts of the
> application could simply access the filesystem directly and do
> whatever modifications it wants. Those notifications will
> automatically be noticed and synchronized to the server by the worker.
>
Hell yes!!! +1000 to add an interface to inotify :-D

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Mon, Jul 22, 2013 at 11:18 AM, Jan Varga  wrote:
> On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking  wrote:
>>
>> Hi All,
>>
>> Yesterday a few of us at mozilla went through the FileSystem API
>> proposal we previously sent [1] and tightened it up.
>>
>
> It was also pointed out that we should address multi-file locking too.
> One of the options is to make openRead() and openWrite() take a sequence.
>
> interface Directory {
>   Promise openRead((DOMString or File) file);
>   Promise openWrite((DOMString or File) file,
> OpenWriteOptions options);
>   Promise> openRead(sequence<(DOMString or File)>
> files);
>   Promise> openWrite(sequence<(DOMString or
> File)> files, OpenWriteOptions options);
> }
>
> So, this works with the current proposal, Jonas has a more complex solution
> that probably requires bigger changes in the proposed file system API.
> I'll let Jonas to describe it.

First of all, I'm not sure that multi-file-locking is something that
we need to solve in this API at all. At least not yet. The google API
doesn't support locking of any type (as far as I can tell), so I think
starting with locking on a single-file basis is a better and simpler
place to start.

The main concern I have with this approach is that it creates a very
specific API which solves a pretty narrow problem of copying data
between two files. It doesn't support copying data between a server
and a file, or between indexedDB and a file.

I can think of two more generic solutions that will solve these problems:

A) Adding something like inotify.
I.e. add the ability to get notifications about changes to part of a
filesystem. This way a page could create a lock-file when it wants to
prevent others from accessing a particular part of the filesystem.

If the lockfile already exists, it could register to be notified when
the lockfile is removed. Once the file is removed it would create the
file and start accessing the files in whatever way it wants.

This is how multi-process access to application data is often handled
in filesystems today.

Adding something like inotify would also enable use cases like having
a worker synchronize a filesystem to a server. Other parts of the
application could simply access the filesystem directly and do
whatever modifications it wants. Those notifications will
automatically be noticed and synchronized to the server by the worker.

B) Add a generic "cross-window lock" mechanism.

Basically an API for asynchronously requesting a lock. Once the lock
becomes available the caller would be notified. When the caller is
done using the lock, he/she calls a function to explicitly release the
lock, thus enabling other callers to get notified that they are now
holding the lock.

A lock is never automatically released by the platform on some timeout
or similar. The only time the lock is forcefully released is when the
user closes the page that created the lock.

It is the application's responsibility to determine when it is
appropriate to grab the lock. I.e. there is no platform connection
between a lock and the resources that it protects. This way a lock
could represent anything from the whole filesystem, a couple of IDB
database and some server resources, to just parts of a file.

Darin Fisher has proposed something similar in the past as I recall
it. Though the details might not exactly match the above.


Pros of A
- Solves more use cases than simply locking-related ones. We've
discussed adding similar observer mechanisms to IDB because the same
non-locking-related usecases have come up there.
- Enables building something like solution B on top of it.

Cons of A
- More complicated to use correctly. For example it's important to
register for the notification before checking if the lock file is
already created. Otherwise there's a risk that the lock file is
removed between the time when the page checks if its there and the
notification is registered.

Pros of B
- Easier to use than A

Cons of B
- Doesn't help with use cases other than locking. I.e. to enable one
window to notice when another window modified some data, you would
either have to use polling, or create custom signaling mechanisms and
make sure to use those whenever something in the filesystem is
modified.

/ Jonas

Re: Polished FileSystem API proposal

[Jan Varga]

And here's an example:

// Copy the 1th byte of file1.bin to file2.bin
navigator.getFilesystem().then(function(root) {
  return root.openWrite(["file1.bin", "file2.bin"]);
}).then(function(handles) {
  return handles[0].read(1);
}).then(function(buffer) {
  return handles[1].write(buffer);
});

On Mon, Jul 22, 2013 at 8:18 PM, Jan Varga  wrote:

> On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking  wrote:
>
>> Hi All,
>>
>> Yesterday a few of us at mozilla went through the FileSystem API
>> proposal we previously sent [1] and tightened it up.
>>
>>
> It was also pointed out that we should address multi-file locking too.
> One of the options is to make openRead() and openWrite() take a sequence.
>
> interface Directory {
>   Promise openRead((DOMString or File) file);
>Promise openWrite((DOMString or File) file,
> OpenWriteOptions options);
>   Promise> openRead(sequence<(DOMString or File)>
> files);
>   Promise> openWrite(sequence<(DOMString or
> File)> files, OpenWriteOptions options);
> }
>
> So, this works with the current proposal, Jonas has a more complex
> solution that probably requires bigger changes in the proposed file system
> API.
> I'll let Jonas to describe it.
>
> Jan
>
>

Re: Polished FileSystem API proposal

[Jan Varga]

On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking  wrote:

> Hi All,
>
> Yesterday a few of us at mozilla went through the FileSystem API
> proposal we previously sent [1] and tightened it up.
>
>
It was also pointed out that we should address multi-file locking too.
One of the options is to make openRead() and openWrite() take a sequence.

interface Directory {
  Promise openRead((DOMString or File) file);
  Promise openWrite((DOMString or File) file,
OpenWriteOptions options);
  Promise> openRead(sequence<(DOMString or File)>
files);
  Promise> openWrite(sequence<(DOMString or
File)> files, OpenWriteOptions options);
}

So, this works with the current proposal, Jonas has a more complex solution
that probably requires bigger changes in the proposed file system API.
I'll let Jonas to describe it.

Jan

Re: Polished FileSystem API proposal

[Jonas Sicking]

Or simply if we make it possible to pass a Directory through
postMessage then you can open another website in a sandboxed 
and pass it a Directory and let it modify its contents, without having
to grant access to other parts of the sandboxed filesystem.

/ Jonas

On Thu, Jul 18, 2013 at 5:02 PM, Anne van Kesteren  wrote:
> On Thu, Jul 18, 2013 at 4:52 PM, Mounir Lamouri  wrote:
>> Isn't a Directory object just a path? I mean, would you send the list of
>> files and their content or just the path to the directory? If you
>> literally want to pass the Directory object, I am not sure how passing
>> the path is different except that you would have to know if this is
>> temporary or permanent storage.
>
> If you use a sandboxed system like Caja you could imagine passing the
> Directory and thereby not exposing any of the files higher in the
> hierarchy. However, Caja could probably also work around this somehow
> given different primitives.
>
>
> --
> http://annevankesteren.nl/
>

Re: Polished FileSystem API proposal

[Anne van Kesteren]

On Thu, Jul 18, 2013 at 4:52 PM, Mounir Lamouri  wrote:
> Isn't a Directory object just a path? I mean, would you send the list of
> files and their content or just the path to the directory? If you
> literally want to pass the Directory object, I am not sure how passing
> the path is different except that you would have to know if this is
> temporary or permanent storage.

If you use a sandboxed system like Caja you could imagine passing the
Directory and thereby not exposing any of the files higher in the
hierarchy. However, Caja could probably also work around this somehow
given different primitives.


--
http://annevankesteren.nl/

Re: Polished FileSystem API proposal

[Mounir Lamouri]

On 15/07/13 23:26, Kinuko Yasuda wrote:
> OTOH one limitation I could think of in not having JS object is it'll
> disallow a possible future API expansion for sending a 'Directory'
> object to another app by postMessage.  (It's another popular request we
> get in Chrome)

Isn't a Directory object just a path? I mean, would you send the list of
files and their content or just the path to the directory? If you
literally want to pass the Directory object, I am not sure how passing
the path is different except that you would have to know if this is
temporary or permanent storage.

--
Mounir

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Tue, Jul 16, 2013 at 8:53 AM, Eric U  wrote:
> On Tue, Jul 16, 2013 at 12:32 AM, Jonas Sicking  wrote:
>> On Mon, Jul 15, 2013 at 11:02 PM, Kinuko Yasuda  wrote:
>>> Glad to see this proposal has a new draft.
>>>
>>> On Sat, Jul 13, 2013 at 9:31 AM, Jonas Sicking  wrote:
>>>>
>>>> Executive Summary (aka TL;DR):
>>>> Below is the mozilla proposal for a simplified filesystem API. It
>>>> contains two new abstractions, a Directory object which allows
>>>> manipulating files and directories within it, and a FileHandle object
>>>> which allows holding an exclusive lock on a file while performing
>>>> multiple read/write operations on it.
>>>>
>>>> It's largely modeled after posix, but because we've tried to keep it
>>>> author friendly despite it's asynchronous nature, it differs in a few
>>>> cases.
>>>>
>>>> There are opportunities for further simplifications by straying
>>>> further from posix. It's unclear if this is desired or not.
>>>>
>>>> Detailed proposal:
>>>>
>>>> partial interface Navigator {
>>>>   Promise getFilesystem(optional FilesystemParameters
>>>> parameters);
>>>> };
>>>>
>>>> interface Directory {
>>>>   readonly attribute DOMString name;
>>>
>>>
>>> So neither File nor Directory has 'path' attribute but only exposes 'name'.
>>> It feels a bit inconvenient but is it intentional?
>>
>> It's something that I intended to add but forgot.
>
> Having the full path within the filesystem removes some of the
> security benefit of not allowing "..", doesn't it?  Conversely, is the
> path useful if you can't use it to manipulate anything above the
> current dir in the tree?

It removes some of the privacy benefits, but little of the security benefits.

It would at least be useful to let something like:

rootdir.get("foo/myfile.html").then(function(file) {
  assert(file.path + file.name === "foo/myfile.html");
});

The tricky thing is what to do if someone calls subdir.get(). Does
that return a File with .path set to a path relative to the root, or a
path relative to the subdir? If relative to the root, it does lose
some of the privacy benefits, but I'm not sure how important that is
within the sandbox. If relative to the subdir that means that you can
get two File objects representing the same file but with two different
paths. That's somewhat confusing.

Also note that we'd definitely never expose a path that goes outside
of the sandbox. I.e. the drag'n'drop or  APIs should
never produce paths that contain directory names outside of the
directories that the user has attached to the page.

>> Keeping .name as just containing the leafname is probably the right thing to 
>> do.
>>
>> So we could introduce a .path property which contains the path within
>> the filesystem. The full filename would be .path + .name.
>>
>> And doing exactly the same for both Directory and File objects seems
>> like a good thing.
>>
>>> File object is meant to be a snapshot and becomes invalid after modification
>>> is made.
>>> Will Directory follow the same model?
>>> Say, if a Directory is acquired then moved (renamed) to another name, does
>>> the Directory object keep functioning or become invalid?
>>
>> Good question. I think the most sane way to implement the Directory
>> object is to internally keep a full path. Whenever an operation is to
>> be performed, we check if that full path still exists. If it doesn't,
>> the operation fails.
>
> Just to be clear: if you rename a Directory's parent, it goes stale
> and you can't use it.  But if you then create a new directory with the
> same name as the parent used to have, with a child of the right name,
> this object will again be valid.  The Directory is explicitly defined
> as referring to a specific path string.

Agreed.

/ Jonas

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Tue, Jul 16, 2013 at 8:56 AM, Eric U  wrote:
> On Mon, Jul 15, 2013 at 5:58 PM, Jonas Sicking  wrote:
>> On Mon, Jul 15, 2013 at 4:47 PM, Eric U  wrote:
> What's the purpose of DestinationDict?  It seems redundant.

 There's basically three potential ways to specify a copy/move destination:

 Specify a new name within the current directory.
 Specify a new directory but keep the current name.
 Specify a new directory as well as a new name within that directory.

 We obviously don't have to support them all, for example you could
 imagine requiring a two-step process in some cases. But we decided to
 try to support all three as a single operation for now.

 Coming up with a syntax for this is somewhat tricky. In C++ you'd
 simply have three overloads that take the different combinations, but
 that doesn't feel very "javascripty".

 Another is to use optional arguments like:

 move(..., Directory? destDir, optional DOMString destName);

 would work. That means that the three options would look like:

 dir.move("srcfile", null, "newName");
 dir.move("srcfile", destDir);
 dir.move("srcfile", destDir, "newName");
>>>
>>> dir.move("srcfile", "dir/and/new/name");
>>>
>>> Granted, that doesn't help you if you're holding a Directory object
>>> and don't have a way to generate a relative path to it. But perhaps
>>> that's not a big lack?  You can always munge paths yourself.
>>>
>>> Obviously we went with the way you'd see as iffy.
>>>
 Unfortunately the first one there looks pretty iffy. So instead we
 went with a union which results in syntax like:

 dir.move("srcfile", "newName");
 dir.move("srcfile", destDir);
 dir.move("srcfile", { dir: destDir. name: "newName" });
>>>
>>> In this syntax, could you say it this way?
>>>
>>> dir.move("srcfile", { dir: "dirName". name: "newName" });
>>
>> No, but you could say
>>
>> dir.move("srcfile", "dirName/newName");
>>
>> Or you can use .get() to grab the directory object named "dirName" and use 
>> that.
>
> So what can't you do without the DestinationDict?  You can rename the
> file without moving it to a new dir, you can move it to a new dir
> without renaming it, and you can rename and move it using
> "dirName/newName" as the second argument.

Only if the new destination directory is a subdir of the current directory.

/ Jonas

> You have MakeFileOptions.overwriteIfExists for createFile, but
> CreateMode { createIfNeeded | dontcreate } for openWrite; it feels
> like those should share a set of flags.  The options should probably
> be {createIfNeeded, dontCreate, failIfExists}, where failIfExists
> implies createIfNeeded and is the standard POSIX paradigm
> [O_CREAT|O_EXCL].  It's useful for coordination between multiple pages
> in the same origin, or the same page loaded twice.

 So you're basically proposing (modulo naming):

 createFile("name", { mode: "createIfNeeded" });
 Creates the file if it doesn't exist. Overwrites it if it does.
>>>
>>> Is the idea that createFile will overwrite an existing file's
>>> contents, even if you don't supply any, e.g. nuking a file down to 0
>>> bytes?
>>
>> Yes.
>>
>>> That might be a bit surprising.  Would you require
>>> MakeFileOptions.data?
>>
>> In the current proposal the function is called "createFile" and if you
>> want it to overwrite what's there you have to explicitly opt in to
>> that using
>>
>> createFile("name", { overwriteIfExists: true });
>>
>> so I don't think the current proposal is that surprising. Especially
>> since creating files usually will either error or remove what's there.
>> I'd like to keep that clarity.
>>
 createFile("name", { mode: "dontCreate" });
 Error if the file doesn't exist. Overwrites it if it does.
>>>
>>> I'm not sure why you'd ever call it this way.  Perhaps disallow it for
>>> now?  Otherwise it might become a standard hack for some bizarre
>>> file-based locking paradigm.
>>>
 createFile("name", { mode: "failIfExists" });
 Creates the file if it doesn't exist. Error it if it does.

 openWrite("name", "createIfNeeded");
 Creates and opens the file if it doesn't exist. Opens it if it does.

 openWrite("name", "dontCreate");
 Error if the file doesn't exist. Opens it if it does.

 openWrite("name", "failIfExists");
 Creates and opens the file if it doesn't exist. Error it if it does.

 Is this correct? While I agree it's consistent, the number of
 combinations here is a bit mind boggling. It took me quite a while to
 write all of these out and make sure they are correct. The
 createFile("dontCreate") combination is especially confusing.
>>>
>>> I'd skip that one; I think the other 5 make sense, and are better than
>>> having two different paradigms to say almost the same thing depending
>>> on which function you call [boolean vs. string argument].

Re: Polished FileSystem API proposal

[pira...@gmail.com]

> Having the full path within the filesystem removes some of the
> security benefit of not allowing "..", doesn't it?
>
There's no problem if it's just an implementation detail.

> Conversely, is the
> path useful if you can't use it to manipulate anything above the
> current dir in the tree?

It's useful to isolate it as a sandbox. Maybe we are talking about
"mount points" that expose the root of a user selected filesystem
subtree (I hope answer is yes... :-) )?


--
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Mon, Jul 15, 2013 at 11:02 PM, Kinuko Yasuda  wrote:
> Glad to see this proposal has a new draft.
>
> On Sat, Jul 13, 2013 at 9:31 AM, Jonas Sicking  wrote:
>>
>> Executive Summary (aka TL;DR):
>> Below is the mozilla proposal for a simplified filesystem API. It
>> contains two new abstractions, a Directory object which allows
>> manipulating files and directories within it, and a FileHandle object
>> which allows holding an exclusive lock on a file while performing
>> multiple read/write operations on it.
>>
>> It's largely modeled after posix, but because we've tried to keep it
>> author friendly despite it's asynchronous nature, it differs in a few
>> cases.
>>
>> There are opportunities for further simplifications by straying
>> further from posix. It's unclear if this is desired or not.
>>
>> Detailed proposal:
>>
>> partial interface Navigator {
>>   Promise getFilesystem(optional FilesystemParameters
>> parameters);
>> };
>>
>> interface Directory {
>>   readonly attribute DOMString name;
>
>
> So neither File nor Directory has 'path' attribute but only exposes 'name'.
> It feels a bit inconvenient but is it intentional?

It's something that I intended to add but forgot.

Keeping .name as just containing the leafname is probably the right thing to do.

So we could introduce a .path property which contains the path within
the filesystem. The full filename would be .path + .name.

And doing exactly the same for both Directory and File objects seems
like a good thing.

> File object is meant to be a snapshot and becomes invalid after modification
> is made.
> Will Directory follow the same model?
> Say, if a Directory is acquired then moved (renamed) to another name, does
> the Directory object keep functioning or become invalid?

Good question. I think the most sane way to implement the Directory
object is to internally keep a full path. Whenever an operation is to
be performed, we check if that full path still exists. If it doesn't,
the operation fails.

>>   Promise createFile(DOMString path, MakeFileOptions options);
>>   Promise createDirectory(DOMString path);
>>
>>   Promise<(File or Directory)> get(DOMString path);
>>
>>   Promise move((DOMString or File or Directory) entry,
>>  (DOMString or Directory or DestinationDict) dest);
>>   Promise copy((DOMString or File or Directory) entry,
>>  (DOMString or Directory or DestinationDict) dest);
>>   Promise remove((DOMString or File or Directory) path,
>>optional DeleteMode recursive = "nonrecursive");
>
>
> One of the major requests we've gotten in Chrome's FS API for potentially
> long-running copy/remove operations is the ability to get progress and to
> abort the operation.
> Does it make sense to make them return AbortableProgressPromise when they
> run recursively?

Yeah, I think that's a good idea. We have to figure out exactly what
the progress information would look like, but that seems very doable.

> It's possible that an app creates tons of files in a directory and then
> copies the entire directory to another, which could take long time
> (depending on implementation).
>
> (Also +1 to have a separate method for recursiveRemove or removeDepp)

Done. Though I'd still like to remove the non-recursive variant. Is it
really useful?

>>   Promise openRead((DOMString or File) file);
>>   Promise openWrite((DOMString or File) file,
>> optional CreateMode createMode = "createifneeded");
>>   Promise openAppend((DOMString or File) file,
>> optional CreateMode createMode = "createifneeded");
>>
>>   EventStream<(File or Directory)> enumerate();
>>   EventStream enumerateDeep();
>
>
> Can this enumeration/stream be stopped halfway?  If a directory contains
> thousands of files callers may not want to keep the disk spinning until the
> enumeration reaches the end.

EventStream is something that still needs to be defined. And shouldn't
be a filesystem specific interface. But yes, it definitely needs to be
stoppable halfway. Maybe we'll need something like
AbortableEventStream, or PausableEventStream, or some such.

>> The FileHandle class automatically closes itself as soon as the page
>> stops posting further calls to .read/.readBinary/.write to it. This
>> happens once the last Promise returned from one of those operations
>> has been resolved, without further calls to .read/.readBinary/.write
>> having happened.
>
> Can the same FileHandle be reused after it's closed, or one need to create a
> new FileHandler to start another sequence of read/write?

Once it's closed, i.e. once the last accept/reject callback has
happened and no new operations were started, the FileHandle becomes
useless. You have to call openRead/openWrite again to start another
sequence.

/ Jonas

Re: Polished FileSystem API proposal

[Kinuko Yasuda]

On Tue, Jul 16, 2013 at 11:29 AM, Mounir Lamouri  wrote:

> Hi,
>
> I am not a big fan of the "Directory" approach of this proposal. It puts
> the API between a high level, object oriented API and a low level API.
> It is unfortunately not really high level because you have to use the
> Directory for most operations and the File objects can't be subject to
> any action.
>

I don't have a strong opinion on this but I kinda agree that introducing a
'Directory' object may not be definitely necessary in this API.
 Associating a JS object to a file or a directory could easily result in a
stale object, and 'snapshot' nature often confuses developers.  Looking at
other API examples, Node.js's FileSystem API purely works on path names and
it looks much simpler. http://nodejs.org/api/fs.html

OTOH one limitation I could think of in not having JS object is it'll
disallow a possible future API expansion for sending a 'Directory' object
to another app by postMessage.  (It's another popular request we get in
Chrome)

I designed an API that looks very similar to the one proposed here but
> based around DOMString instead of File and Directory. This API is meant
> to be low level and more basic/simple. It gives room for JS libraries to
> come up with their own high level object oriented API design and adds
> very little overhead for simple use case.
>
> This API is built around a FileSystem interface that exposes all the
> methods needed to manipulate the virtual filesystem. Getting a
> FileSystem object would be done with:
>  navigator.getFileSystem({"temporary", "permanent"}).
> An alternative would be to have navigator.filesystem that would return a
> FileSystem object and navigator.requestPermanentFileSystem() that could
> be used to make the navigator.filesystem object different (ie. permanent
> vs temporary). That approach would make using temporary storage more
> straightforward but could make using permanent storage a bit more painful.
>
> This API is a bit less polished than the initial proposal (there were
> less eyes on it) and contrary to the initial proposal, it is trying to
> be very simple regarding the behaviour. For example, .remove() would
> remove the directory/file and all sub-directories. The consumer of the
> API is expected to check if the directory is empty before calling the
> method. The API is re-using FileHandle and FileHandleWritable.
>
> There are a couple of functions there only to make developers' life
> easier assuming they would make a common use case simpler. Removing them
> is definitely an option.
>
> The interface below describes this alternative proposal. The comments
> should give a rough explanation of what each method is doing.
>
> enum FileSystemType {
>   "temporary",
>   "persistent"
> };
>
> partial interface Navigator {
>   FileSystem getFileSystem(optional FileSystemType type = "temporary");
> };
>
> interface FileSystem {
>   // Those two methods will create the directory/file if it does not
>   // exist but will keep there content as is. The returned value will
>   // be whether something as been created.
>   Promise createDir(DOMString path);
>   Promise createFile(DOMString path);
>
>   // Can be used to rename or move. Whether for a file or directory.
>   Promise move(DOMString source, DOMString destination);
>
>   // Mostly to make developer's life easier.
>   Promise copy(DOMString source, DOMString destination);
>
>   // Removes the file if the path is a file or the sub-tree if the path
>   // is a directory.
>   Promise remove(DOMString path);
>
>   // The next three functions would reject if the path is a directory.
>   Promise read(DOMString path);
>   Promise write(DOMString path);
>   // This is for convenience so we can easily write a Blob.
>   Promise write(DOMString path, Blob data);
>
>   // TODO: we could add append().
>   // For convenience. Could be expressed as:
>   // .read(path).then(f) { return f.getFile(); }
>   Promise getFile(DOMString path);
>
>   EventStream enumerate(DOMString path, optional boolean deep
> = false);
>
>   // Those two methods are trivial. They could be merged into one
>   // method that would return a tri-state enum: "file", "directory",
>   // "notfound".
>   Promise isDirectory(DOMString path);
>   Promise exists(DOMString path);
>
>   // TODO: we could add something like .childrenCount that would return
>   // how many files/directory a directory contains. Would be 0 for a
>   // file. This could also be .isEmpty(). But that might be
>   // mis-interpreted for files.
> };
>
> Thanks,
> --
> Mounir
>

Re: Polished FileSystem API proposal

[Kinuko Yasuda]

Glad to see this proposal has a new draft.

On Sat, Jul 13, 2013 at 9:31 AM, Jonas Sicking  wrote:

> Executive Summary (aka TL;DR):
> Below is the mozilla proposal for a simplified filesystem API. It
> contains two new abstractions, a Directory object which allows
> manipulating files and directories within it, and a FileHandle object
> which allows holding an exclusive lock on a file while performing
> multiple read/write operations on it.
>
> It's largely modeled after posix, but because we've tried to keep it
> author friendly despite it's asynchronous nature, it differs in a few
> cases.
>
> There are opportunities for further simplifications by straying
> further from posix. It's unclear if this is desired or not.
>
> Detailed proposal:
>
> partial interface Navigator {
>   Promise getFilesystem(optional FilesystemParameters
> parameters);
> };
>
> interface Directory {
>   readonly attribute DOMString name;
>

So neither File nor Directory has 'path' attribute but only exposes 'name'.
It feels a bit inconvenient but is it intentional?

File object is meant to be a snapshot and becomes invalid after
modification is made.
Will Directory follow the same model?
Say, if a Directory is acquired then moved (renamed) to another name, does
the Directory object keep functioning or become invalid?


>   Promise createFile(DOMString path, MakeFileOptions options);
>   Promise createDirectory(DOMString path);
>
>   Promise<(File or Directory)> get(DOMString path);
>
>   Promise move((DOMString or File or Directory) entry,
>  (DOMString or Directory or DestinationDict) dest);
>   Promise copy((DOMString or File or Directory) entry,
>  (DOMString or Directory or DestinationDict) dest);
>   Promise remove((DOMString or File or Directory) path,
>optional DeleteMode recursive = "nonrecursive");
>

One of the major requests we've gotten in Chrome's FS API for potentially
long-running copy/remove operations is the ability to get progress and to
abort the operation.
Does it make sense to make them return AbortableProgressPromise when they
run recursively?

It's possible that an app creates tons of files in a directory and then
copies the entire directory to another, which could take long time
(depending on implementation).

(Also +1 to have a separate method for recursiveRemove or removeDepp)

  Promise openRead((DOMString or File) file);
>   Promise openWrite((DOMString or File) file,
> optional CreateMode createMode = "createifneeded");
>   Promise openAppend((DOMString or File) file,
> optional CreateMode createMode = "createifneeded");
>
>   EventStream<(File or Directory)> enumerate();
>   EventStream enumerateDeep();
>

Can this enumeration/stream be stopped halfway?  If a directory contains
thousands of files callers may not want to keep the disk spinning until the
enumeration reaches the end.

};
>
> interface FileHandle
> {
>   readonly attribute FileOpenMode mode;
>   readonly attribute boolean active;
>
>   attribute long long? location;
>
>   Promise getFile();
>   AbortableProgressPromise read(unsigned long long size);
>   AbortableProgressPromise readText(unsigned long long
> size, optional DOMString encoding = "utf-8");
>
>   void abort();
> };
>
> interface FileHandleWritable : FileHandle
> {
>   AbortableProgressPromise write((DOMString or ArrayBuffer or
> ArrayBufferView or Blob) value);
>
>   Promise setSize(optional unsigned long long size);
>
>   Promise flush();
> };
>
> partial interface URL {
>   static DOMString? getPersistentURL(File file);
> }
>
> // WebIDL cruft that's largely transparent
> enum PersistenceType { "temporary", "persistent" };
> dictionary FilesystemParameters {
>   PersistenceType storage = "temporary";
> };
>
> dictionary MakeFileOptions {
>   boolean overwriteIfExists = false;
>

The term 'overwrite' feels a bit confusing as it could mean 'overwriting a
file without truncating'.
If what it really means is truncate-if-exists can we rename this to
'truncateIfExists' (or { ifExists: "truncate" }) so that it becomes clearer
it works just like O_TRUNC?

  (DOMString or Blob or ArrayBuffer or ArrayBufferView) data;
> };
>
> enum CreateMode { "createifneeded", "dontcreate" }
> enum DeleteMode { "recursive", "nonrecursive" }
>
> dictionary DestinationDict {
>   Directory dir;
>   DOMString name;
> };
>
> enum FileOpenMode { "read", "write", "append" };
>
> So this AP

Re: Polished FileSystem API proposal

[Mounir Lamouri]

Hi,

I am not a big fan of the "Directory" approach of this proposal. It puts
the API between a high level, object oriented API and a low level API.
It is unfortunately not really high level because you have to use the
Directory for most operations and the File objects can't be subject to
any action.

I designed an API that looks very similar to the one proposed here but
based around DOMString instead of File and Directory. This API is meant
to be low level and more basic/simple. It gives room for JS libraries to
come up with their own high level object oriented API design and adds
very little overhead for simple use case.

This API is built around a FileSystem interface that exposes all the
methods needed to manipulate the virtual filesystem. Getting a
FileSystem object would be done with:
 navigator.getFileSystem({"temporary", "permanent"}).
An alternative would be to have navigator.filesystem that would return a
FileSystem object and navigator.requestPermanentFileSystem() that could
be used to make the navigator.filesystem object different (ie. permanent
vs temporary). That approach would make using temporary storage more
straightforward but could make using permanent storage a bit more painful.

This API is a bit less polished than the initial proposal (there were
less eyes on it) and contrary to the initial proposal, it is trying to
be very simple regarding the behaviour. For example, .remove() would
remove the directory/file and all sub-directories. The consumer of the
API is expected to check if the directory is empty before calling the
method. The API is re-using FileHandle and FileHandleWritable.

There are a couple of functions there only to make developers' life
easier assuming they would make a common use case simpler. Removing them
is definitely an option.

The interface below describes this alternative proposal. The comments
should give a rough explanation of what each method is doing.

enum FileSystemType {
  "temporary",
  "persistent"
};

partial interface Navigator {
  FileSystem getFileSystem(optional FileSystemType type = "temporary");
};

interface FileSystem {
  // Those two methods will create the directory/file if it does not
  // exist but will keep there content as is. The returned value will
  // be whether something as been created.
  Promise createDir(DOMString path);
  Promise createFile(DOMString path);

  // Can be used to rename or move. Whether for a file or directory.
  Promise move(DOMString source, DOMString destination);

  // Mostly to make developer's life easier.
  Promise copy(DOMString source, DOMString destination);

  // Removes the file if the path is a file or the sub-tree if the path
  // is a directory.
  Promise remove(DOMString path);

  // The next three functions would reject if the path is a directory.
  Promise read(DOMString path);
  Promise write(DOMString path);
  // This is for convenience so we can easily write a Blob.
  Promise write(DOMString path, Blob data);

  // TODO: we could add append().
  // For convenience. Could be expressed as:
  // .read(path).then(f) { return f.getFile(); }
  Promise getFile(DOMString path);

  EventStream enumerate(DOMString path, optional boolean deep
= false);

  // Those two methods are trivial. They could be merged into one
  // method that would return a tri-state enum: "file", "directory",
  // "notfound".
  Promise isDirectory(DOMString path);
  Promise exists(DOMString path);

  // TODO: we could add something like .childrenCount that would return
  // how many files/directory a directory contains. Would be 0 for a
  // file. This could also be .isEmpty(). But that might be
  // mis-interpreted for files.
};

Thanks,
--
Mounir

On 12/07/13 17:31, Jonas Sicking wrote:
> Hi All,
> 
> Yesterday a few of us at mozilla went through the FileSystem API
> proposal we previously sent [1] and tightened it up.
> 
> Executive Summary (aka TL;DR):
> Below is the mozilla proposal for a simplified filesystem API. It
> contains two new abstractions, a Directory object which allows
> manipulating files and directories within it, and a FileHandle object
> which allows holding an exclusive lock on a file while performing
> multiple read/write operations on it.
> 
> It's largely modeled after posix, but because we've tried to keep it
> author friendly despite it's asynchronous nature, it differs in a few
> cases.
> 
> There are opportunities for further simplifications by straying
> further from posix. It's unclear if this is desired or not.
> 
> Detailed proposal:
> 
> partial interface Navigator {
>   Promise getFilesystem(optional FilesystemParameters parameters);
> };
> 
> interface Directory {
>   readonly attribute DOMString name;
> 
>   Promise createFile(DOMString path, MakeFileOptions options);

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Mon, Jul 15, 2013 at 4:47 PM, Eric U  wrote:
>>> What's the purpose of DestinationDict?  It seems redundant.
>>
>> There's basically three potential ways to specify a copy/move destination:
>>
>> Specify a new name within the current directory.
>> Specify a new directory but keep the current name.
>> Specify a new directory as well as a new name within that directory.
>>
>> We obviously don't have to support them all, for example you could
>> imagine requiring a two-step process in some cases. But we decided to
>> try to support all three as a single operation for now.
>>
>> Coming up with a syntax for this is somewhat tricky. In C++ you'd
>> simply have three overloads that take the different combinations, but
>> that doesn't feel very "javascripty".
>>
>> Another is to use optional arguments like:
>>
>> move(..., Directory? destDir, optional DOMString destName);
>>
>> would work. That means that the three options would look like:
>>
>> dir.move("srcfile", null, "newName");
>> dir.move("srcfile", destDir);
>> dir.move("srcfile", destDir, "newName");
>
> dir.move("srcfile", "dir/and/new/name");
>
> Granted, that doesn't help you if you're holding a Directory object
> and don't have a way to generate a relative path to it. But perhaps
> that's not a big lack?  You can always munge paths yourself.
>
> Obviously we went with the way you'd see as iffy.
>
>> Unfortunately the first one there looks pretty iffy. So instead we
>> went with a union which results in syntax like:
>>
>> dir.move("srcfile", "newName");
>> dir.move("srcfile", destDir);
>> dir.move("srcfile", { dir: destDir. name: "newName" });
>
> In this syntax, could you say it this way?
>
> dir.move("srcfile", { dir: "dirName". name: "newName" });

No, but you could say

dir.move("srcfile", "dirName/newName");

Or you can use .get() to grab the directory object named "dirName" and use that.

>>> You have MakeFileOptions.overwriteIfExists for createFile, but
>>> CreateMode { createIfNeeded | dontcreate } for openWrite; it feels
>>> like those should share a set of flags.  The options should probably
>>> be {createIfNeeded, dontCreate, failIfExists}, where failIfExists
>>> implies createIfNeeded and is the standard POSIX paradigm
>>> [O_CREAT|O_EXCL].  It's useful for coordination between multiple pages
>>> in the same origin, or the same page loaded twice.
>>
>> So you're basically proposing (modulo naming):
>>
>> createFile("name", { mode: "createIfNeeded" });
>> Creates the file if it doesn't exist. Overwrites it if it does.
>
> Is the idea that createFile will overwrite an existing file's
> contents, even if you don't supply any, e.g. nuking a file down to 0
> bytes?

Yes.

> That might be a bit surprising.  Would you require
> MakeFileOptions.data?

In the current proposal the function is called "createFile" and if you
want it to overwrite what's there you have to explicitly opt in to
that using

createFile("name", { overwriteIfExists: true });

so I don't think the current proposal is that surprising. Especially
since creating files usually will either error or remove what's there.
I'd like to keep that clarity.

>> createFile("name", { mode: "dontCreate" });
>> Error if the file doesn't exist. Overwrites it if it does.
>
> I'm not sure why you'd ever call it this way.  Perhaps disallow it for
> now?  Otherwise it might become a standard hack for some bizarre
> file-based locking paradigm.
>
>> createFile("name", { mode: "failIfExists" });
>> Creates the file if it doesn't exist. Error it if it does.
>>
>> openWrite("name", "createIfNeeded");
>> Creates and opens the file if it doesn't exist. Opens it if it does.
>>
>> openWrite("name", "dontCreate");
>> Error if the file doesn't exist. Opens it if it does.
>>
>> openWrite("name", "failIfExists");
>> Creates and opens the file if it doesn't exist. Error it if it does.
>>
>> Is this correct? While I agree it's consistent, the number of
>> combinations here is a bit mind boggling. It took me quite a while to
>> write all of these out and make sure they are correct. The
>> createFile("dontCreate") combination is especially confusing.
>
> I'd skip that one; I think the other 5 make sense, and are better than
> having two different paradigms to say almost the same thing depending
> on which function you call [boolean vs. string argument].  If you
> decide you don't want failIfExists, that makes the list even simpler.

I think the 4 modes that are important to support are:

Create new file and error if there's already an existing one
Create new file and overwrite if there's already an existing one
Open for writing and create if doesn't exist
Open for writing and error if doesn't exist

The behavior that needs to be defined for the create function is what
to do if the file already exists, because generally you aren't
expecting a file to exist when you are requesting it to be created.
The behavior that needs to be defined for the open function is what to
do if the function does not exist.

So I don't 

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Mon, Jul 15, 2013 at 2:42 PM, Eric U  wrote:
> 75% bikeshedding, 25% lessons learned:
>
> It seems inconsistent to have the API for recursion done differently
> in two different parts of the API: you've got two functions for
> enumerate/enumerateDeep but remove takes a parameter to be recursive.
> I don't have a strong preference between them, but I think you should
> only use one style.

Good point. I'll change it to remove and removeDeep for now. That also
avoids the "what is default" question.

Out of curiosity, has it been a problem for you that the
DirectoryEntry API doesn't have a non-recursive remove option?
Granted, you do have the ability to grab a sub-Entry and call .remove
on it. We didn't have that option since we didn't want to create an
equivalent of FileEntry.

> What's the purpose of DestinationDict?  It seems redundant.

There's basically three potential ways to specify a copy/move destination:

Specify a new name within the current directory.
Specify a new directory but keep the current name.
Specify a new directory as well as a new name within that directory.

We obviously don't have to support them all, for example you could
imagine requiring a two-step process in some cases. But we decided to
try to support all three as a single operation for now.

Coming up with a syntax for this is somewhat tricky. In C++ you'd
simply have three overloads that take the different combinations, but
that doesn't feel very "javascripty".

Another is to use optional arguments like:

move(..., Directory? destDir, optional DOMString destName);

would work. That means that the three options would look like:

dir.move("srcfile", null, "newName");
dir.move("srcfile", destDir);
dir.move("srcfile", destDir, "newName");

Unfortunately the first one there looks pretty iffy. So instead we
went with a union which results in syntax like:

dir.move("srcfile", "newName");
dir.move("srcfile", destDir);
dir.move("srcfile", { dir: destDir. name: "newName" });

Another solution might be to only use dictionaries. Resulting in

dir.move("srcfile", { name: "newName" });
dir.move("srcfile", { dir: destDir });
dir.move("srcfile", { dir: destDir. name: "newName" });

I think I'd prefer to defer to TC39 about which option is best here.
I'll point this out explicitly when emailing public-script-coord.

> You have MakeFileOptions.overwriteIfExists for createFile, but
> CreateMode { createIfNeeded | dontcreate } for openWrite; it feels
> like those should share a set of flags.  The options should probably
> be {createIfNeeded, dontCreate, failIfExists}, where failIfExists
> implies createIfNeeded and is the standard POSIX paradigm
> [O_CREAT|O_EXCL].  It's useful for coordination between multiple pages
> in the same origin, or the same page loaded twice.

So you're basically proposing (modulo naming):

createFile("name", { mode: "createIfNeeded" });
Creates the file if it doesn't exist. Overwrites it if it does.

createFile("name", { mode: "dontCreate" });
Error if the file doesn't exist. Overwrites it if it does.

createFile("name", { mode: "failIfExists" });
Creates the file if it doesn't exist. Error it if it does.

openWrite("name", "createIfNeeded");
Creates and opens the file if it doesn't exist. Opens it if it does.

openWrite("name", "dontCreate");
Error if the file doesn't exist. Opens it if it does.

openWrite("name", "failIfExists");
Creates and opens the file if it doesn't exist. Error it if it does.

Is this correct? While I agree it's consistent, the number of
combinations here is a bit mind boggling. It took me quite a while to
write all of these out and make sure they are correct. The
createFile("dontCreate") combination is especially confusing.

> If you don't allow ".." to traverse directories, please define using
> it as a path segment as an error, to allow for future expansion and to
> catch bugs for folks who didn't read the spec.

Definitely.

> openForWrite or openForWriting, while longer, would be clearer than
> openWrite.  Heck, why not just "write"?

I tried out just "write" for a while, but it seemed strange that
myDir.write() actually started writing to a file.

> We've had folks ask for progress events on long move and copy calls.
> Think about network drive latencies, big SD cards being brought into
> the browser world, etc.  It's especially important if you think you'll
> ever use this API outside the sandbox, and cheap to add now even if
> you won't.  If you don't have it, people hack around it by
> reimplementing copy using write and createDirectory, and it's tedious
> and error-prone.  In your case, you'll probably want them for
> createFile as well, since you can pass in a gigantic Blob or
> ArrayBuffer there.

Hmm.. Good points. For directory copy/move operations we couldn't
provide a total size, but we could still provide progress in the form
of total number number of bytes copied so far, as well as "currently
copying/moving file with name X".

/ Jonas

Re: Polished FileSystem API proposal

[Janusz Majnert]

15 lip 2013 10:47, "Jonas Sicking"  napisał(a):
>
> On Mon, Jul 15, 2013 at 12:18 AM, Janusz Majnert 
wrote:
> > Hi,
> >
> > On 2013-07-13 02:31, Jonas Sicking wrote:
> >>
> >> [...]
> >>
> >> interface FileHandle
> >> {
> >>readonly attribute FileOpenMode mode;
> >>readonly attribute boolean active;
> >>
> >>attribute long long? location;
> >
> > "location" seems to be a bad name. It might be confused with "location
in
> > the filesystem". Why not use "offset" instead? Is this value 0-based?
>
> That seems better yeah.
>
> >> Do we really need the .openAppend() function? Or is it ok to ask
> >> people to use .openWrite() and then go to the end before writing?
> >
> >
> > In the "append" mode, is the "location" in file changed to the end of
file
> > before every write?
>
> No. It's just initially set to the end of the file. Though since every
> write moves the offset to the end of the write, that means that the
> offset will remain at the end of the file unless explicitly set.
Yes, but if you change the offset to point to the beginnig of the file,
then a subsequent write happens there, unlike in posix, which would first
move the pointer to the eof and then write the data.
IMHO openAppend should be dropped. It will confuse people used to posix
style file access, and for others it's just a shorthand for
openWrite+location=null.

>
> > If not, then I think openAppend should be removed. In
> > the example you gave at the bottom of your message, it seems that it's
> > actually possible to write in the middle of the file.
>
> It is indeed.
>
> > Another non-posix thing is that you can read a file that is in "write"
mode.
> > I would therefore propose to:
> > * remove openAppend()
> > * change enum FileOpenMode to {"read", "readWrite"}
> >
> > In another email you wrote that this API is not meant to be used to
access
> > the OS filesystem, but a "snadboxed" filesystem abstraction for a
webapp. If
> > so, why do we even need the two access modes? Let's just have an
openFile()
> > that returns a File object you can read and write to.
>
> I can think of four reasons, though none of them particularly great.
>
> * While it's not a primary target for this API, it's nice if we can
> reuse the same API if we ever end up introducing APIs that will allow
> accessing real filesystems. Though you could argue that it's ok if you
> in that case too always open the file in readwrite mode.
> * Even in a sandboxed filesystem you could end up with multiple actors
> accessing the filesystem at the same time. For example the user might
> have the same website open in two tabs. Or you might have a worker and
> a page accessing the file at the same time. With openRead you can
> allow multiple simultaneous readers.
> * We might want to support a read-only filesystem in some situations.
> For example if the user uses drag'n'drop to expose a folder to a page,
> we could expose that folder using a read-only version of the Directory
> interface. Though in that case not exposing open* at all and instead
> relying on get() would probably work.
> * A main reason we're considering exposing a filesystem API at all is
> it's a familiar concept to authors. And most filesystem APIs have
> openRead and openWrite (and posix has openAppend).
>
> / Jonas
>

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Mon, Jul 15, 2013 at 12:18 AM, Janusz Majnert  wrote:
> Hi,
>
> On 2013-07-13 02:31, Jonas Sicking wrote:
>>
>> [...]
>>
>> interface FileHandle
>> {
>>readonly attribute FileOpenMode mode;
>>readonly attribute boolean active;
>>
>>attribute long long? location;
>
> "location" seems to be a bad name. It might be confused with "location in
> the filesystem". Why not use "offset" instead? Is this value 0-based?

That seems better yeah.

>> Do we really need the .openAppend() function? Or is it ok to ask
>> people to use .openWrite() and then go to the end before writing?
>
>
> In the "append" mode, is the "location" in file changed to the end of file
> before every write?

No. It's just initially set to the end of the file. Though since every
write moves the offset to the end of the write, that means that the
offset will remain at the end of the file unless explicitly set.

> If not, then I think openAppend should be removed. In
> the example you gave at the bottom of your message, it seems that it's
> actually possible to write in the middle of the file.

It is indeed.

> Another non-posix thing is that you can read a file that is in "write" mode.
> I would therefore propose to:
> * remove openAppend()
> * change enum FileOpenMode to {"read", "readWrite"}
>
> In another email you wrote that this API is not meant to be used to access
> the OS filesystem, but a "snadboxed" filesystem abstraction for a webapp. If
> so, why do we even need the two access modes? Let's just have an openFile()
> that returns a File object you can read and write to.

I can think of four reasons, though none of them particularly great.

* While it's not a primary target for this API, it's nice if we can
reuse the same API if we ever end up introducing APIs that will allow
accessing real filesystems. Though you could argue that it's ok if you
in that case too always open the file in readwrite mode.
* Even in a sandboxed filesystem you could end up with multiple actors
accessing the filesystem at the same time. For example the user might
have the same website open in two tabs. Or you might have a worker and
a page accessing the file at the same time. With openRead you can
allow multiple simultaneous readers.
* We might want to support a read-only filesystem in some situations.
For example if the user uses drag'n'drop to expose a folder to a page,
we could expose that folder using a read-only version of the Directory
interface. Though in that case not exposing open* at all and instead
relying on get() would probably work.
* A main reason we're considering exposing a filesystem API at all is
it's a familiar concept to authors. And most filesystem APIs have
openRead and openWrite (and posix has openAppend).

/ Jonas

Re: Polished FileSystem API proposal

[Janusz Majnert]

Hi,

On 2013-07-13 02:31, Jonas Sicking wrote:

[...]
interface FileHandle
{
   readonly attribute FileOpenMode mode;
   readonly attribute boolean active;

   attribute long long? location;
"location" seems to be a bad name. It might be confused with "location 
in the filesystem". Why not use "offset" instead? Is this value 0-based?




   Promise getFile();
   AbortableProgressPromise read(unsigned long long size);
   AbortableProgressPromise readText(unsigned long long
size, optional DOMString encoding = "utf-8");

   void abort();
};

interface FileHandleWritable : FileHandle
{
   AbortableProgressPromise write((DOMString or ArrayBuffer or
ArrayBufferView or Blob) value);

   Promise setSize(optional unsigned long long size);

   Promise flush();
};

partial interface URL {
   static DOMString? getPersistentURL(File file);
}

// WebIDL cruft that's largely transparent
enum PersistenceType { "temporary", "persistent" };
dictionary FilesystemParameters {
   PersistenceType storage = "temporary";
};

dictionary MakeFileOptions {
   boolean overwriteIfExists = false;
   (DOMString or Blob or ArrayBuffer or ArrayBufferView) data;
};

enum CreateMode { "createifneeded", "dontcreate" }
enum DeleteMode { "recursive", "nonrecursive" }

dictionary DestinationDict {
   Directory dir;
   DOMString name;
};

enum FileOpenMode { "read", "write", "append" };


> [...]
>

Do we really need the .openAppend() function? Or is it ok to ask
people to use .openWrite() and then go to the end before writing?


In the "append" mode, is the "location" in file changed to the end of 
file before every write? If not, then I think openAppend should be 
removed. In the example you gave at the bottom of your message, it seems 
that it's actually possible to write in the middle of the file.


Another non-posix thing is that you can read a file that is in "write" mode.
I would therefore propose to:
* remove openAppend()
* change enum FileOpenMode to {"read", "readWrite"}

In another email you wrote that this API is not meant to be used to 
access the OS filesystem, but a "snadboxed" filesystem abstraction for a 
webapp. If so, why do we even need the two access modes? Let's just have 
an openFile() that returns a File object you can read and write to.




--
Janusz Majnert
Samsung R&D Institute Poland
Samsung Electronics

Re: Polished FileSystem API proposal

[David Rajchenbach-Teller]

Why both createFile, open{Read, Write, Append} and get? Is it to avoid a
signature with dependent types?

Cheers,
 David

On 7/13/13 2:31 AM, Jonas Sicking wrote:
> 
>   Promise createFile(DOMString path, MakeFileOptions options);
>   Promise createDirectory(DOMString path);
> 
>   Promise<(File or Directory)> get(DOMString path);
> 
>   Promise move((DOMString or File or Directory) entry,
>  (DOMString or Directory or DestinationDict) dest);
>   Promise copy((DOMString or File or Directory) entry,
>  (DOMString or Directory or DestinationDict) dest);
>   Promise remove((DOMString or File or Directory) path,
>optional DeleteMode recursive = "nonrecursive");
> 
>   Promise openRead((DOMString or File) file);
>   Promise openWrite((DOMString or File) file,
> optional CreateMode createMode = "createifneeded");
>   Promise openAppend((DOMString or File) file,
> optional CreateMode createMode = "createifneeded");
> 
>   EventStream<(File or Directory)> enumerate();
>   EventStream enumerateDeep();
> };


-- 
David Rajchenbach-Teller, PhD
 Performance Team, Mozilla

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Sat, Jul 13, 2013 at 2:37 AM, Kenneth Rohde Christiansen
 wrote:
> Hi there!
>
> A new file system API with a lot of "promise" :-)
>
> On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking  wrote:
>> Hi All,
>>
>> Yesterday a few of us at mozilla went through the FileSystem API
>> proposal we previously sent [1] and tightened it up.
>>
>> Executive Summary (aka TL;DR):
>> Below is the mozilla proposal for a simplified filesystem API. It
>> contains two new abstractions, a Directory object which allows
>> manipulating files and directories within it, and a FileHandle object
>> which allows holding an exclusive lock on a file while performing
>> multiple read/write operations on it.
>>
>> It's largely modeled after posix, but because we've tried to keep it
>> author friendly despite it's asynchronous nature, it differs in a few
>> cases.
>>
>> There are opportunities for further simplifications by straying
>> further from posix. It's unclear if this is desired or not.
>>
>> Detailed proposal:
>>
>> partial interface Navigator {
>>   Promise getFilesystem(optional FilesystemParameters parameters);
>> };
>>
>> interface Directory {
>>   readonly attribute DOMString name;
>>
>>   Promise createFile(DOMString path, MakeFileOptions options);
>
> Why not CreateFileOptions? the method is called createFile and not makeFile

Yup. I'll fix that.

>>   Promise createDirectory(DOMString path);
>
> You have a data as part of the MakeFileOptions, would that be useful here?

What would it do?

>>   Promise<(File or Directory)> get(DOMString path);
>
> Then shouldn't we have a convenience to see if a path is a directly or
> not, like python has os.path.isdir(fileordirectoryname)

The get method combines the "check if exists", "check if directory",
"stat to get metadata" functions into one. We could certainly add all
three, but it doesn't seem to really make the API easier to use, just
bigger.

>>   Promise move((DOMString or File or Directory) entry,
>>  (DOMString or Directory or DestinationDict) dest);
>>   Promise copy((DOMString or File or Directory) entry,
>>  (DOMString or Directory or DestinationDict) dest);
>>   Promise remove((DOMString or File or Directory) path,
>>optional DeleteMode recursive = "nonrecursive");
>
> I don't like all these weird arguments like DeleteMode etc... just
> make a separate method
>
> removeRecursively(

That's certainly an interesting idea. I'd personally prefer to just
have the recursive variant, but if we really need both then that's
probably better.

>>   Promise openRead((DOMString or File) file);
>>   Promise openWrite((DOMString or File) file,
>> optional CreateMode createMode = "createifneeded");
>
> Can't the user not just handle the creation in the error case of the
> promise. or create it before... It would so very easy to create a
> openWriteCreateIfNeeded method on top of the existing API. So such an
> argument should only be needed if there would be performance benefits
> - otherwise keep the API simple.

Another question is if we need openWrite with support for not creating
the file if it's not there. You could likewise easily also create a
openWriteFailIfDoesntExist on top of existing API by using the get()
function.

I don't feel particularly strongly. Create-and-open-for-writing is
probably a decently common operation. Though createFile probably
covers the most common create-and-open-for-writing scenarios.

>>   Promise openAppend((DOMString or File) file,
>> optional CreateMode createMode = "createifneeded");
>>
>>   EventStream<(File or Directory)> enumerate();
>>   EventStream enumerateDeep();
>
> It is not obvious for me what that method does.

It recursively enumerates all files in this directory and all
subdirectories. This is useful for things like figuring out the total
size of files under a subdirectory, or sending a full directory tree
to the server.

/ Jonas

Re: Polished FileSystem API proposal

[Kenneth Rohde Christiansen]

Hi there!

A new file system API with a lot of "promise" :-)

On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking  wrote:
> Hi All,
>
> Yesterday a few of us at mozilla went through the FileSystem API
> proposal we previously sent [1] and tightened it up.
>
> Executive Summary (aka TL;DR):
> Below is the mozilla proposal for a simplified filesystem API. It
> contains two new abstractions, a Directory object which allows
> manipulating files and directories within it, and a FileHandle object
> which allows holding an exclusive lock on a file while performing
> multiple read/write operations on it.
>
> It's largely modeled after posix, but because we've tried to keep it
> author friendly despite it's asynchronous nature, it differs in a few
> cases.
>
> There are opportunities for further simplifications by straying
> further from posix. It's unclear if this is desired or not.
>
> Detailed proposal:
>
> partial interface Navigator {
>   Promise getFilesystem(optional FilesystemParameters parameters);
> };
>
> interface Directory {
>   readonly attribute DOMString name;
>
>   Promise createFile(DOMString path, MakeFileOptions options);

Why not CreateFileOptions? the method is called createFile and not makeFile

>   Promise createDirectory(DOMString path);

You have a data as part of the MakeFileOptions, would that be useful here?

>   Promise<(File or Directory)> get(DOMString path);

Then shouldn't we have a convenience to see if a path is a directly or
not, like python has os.path.isdir(fileordirectoryname)

>   Promise move((DOMString or File or Directory) entry,
>  (DOMString or Directory or DestinationDict) dest);
>   Promise copy((DOMString or File or Directory) entry,
>  (DOMString or Directory or DestinationDict) dest);
>   Promise remove((DOMString or File or Directory) path,
>optional DeleteMode recursive = "nonrecursive");

I don't like all these weird arguments like DeleteMode etc... just
make a separate method

removeRecursively(

>
>   Promise openRead((DOMString or File) file);
>   Promise openWrite((DOMString or File) file,
> optional CreateMode createMode = "createifneeded");

Can't the user not just handle the creation in the error case of the
promise. or create it before... It would so very easy to create a
openWriteCreateIfNeeded method on top of the existing API. So such an
argument should only be needed if there would be performance benefits
- otherwise keep the API simple.

>   Promise openAppend((DOMString or File) file,
> optional CreateMode createMode = "createifneeded");
>
>   EventStream<(File or Directory)> enumerate();
>   EventStream enumerateDeep();

It is not obvious for me what that method does.

> };
>
> interface FileHandle
> {
>   readonly attribute FileOpenMode mode;
>   readonly attribute boolean active;
>
>   attribute long long? location;
>
>   Promise getFile();
>   AbortableProgressPromise read(unsigned long long size);
>   AbortableProgressPromise readText(unsigned long long
> size, optional DOMString encoding = "utf-8");
>
>   void abort();
> };
>
> interface FileHandleWritable : FileHandle
> {
>   AbortableProgressPromise write((DOMString or ArrayBuffer or
> ArrayBufferView or Blob) value);
>
>   Promise setSize(optional unsigned long long size);
>
>   Promise flush();
> };
>
> partial interface URL {
>   static DOMString? getPersistentURL(File file);
> }
>
> // WebIDL cruft that's largely transparent
> enum PersistenceType { "temporary", "persistent" };

Is temporary a kind of persistence? (sorry not native speaker)

> dictionary FilesystemParameters {
>   PersistenceType storage = "temporary";
> };
>
> dictionary MakeFileOptions {
>   boolean overwriteIfExists = false;
>   (DOMString or Blob or ArrayBuffer or ArrayBufferView) data;
> };
>
> enum CreateMode { "createifneeded", "dontcreate" }
> enum DeleteMode { "recursive", "nonrecursive" }
>
> dictionary DestinationDict {
>   Directory dir;
>   DOMString name;
> };
>
> enum FileOpenMode { "read", "write", "append" };
>
> So this API introduces 2 classes: Directory and FileHandle. Directory
> allows manipulation of the files and directories stored inside that
> directory. FileHandle represents an exclusively opened file and allows
> manipulation of the file contents.
>
> The behavior is hopefully mostly obvious. A few general comments:
>
> The functions on Directory that accept DOMString arguments for
> filenames allow 

Re: Polished FileSystem API proposal

[Jonas Sicking]

On Sat, Jul 13, 2013 at 1:27 AM, David Rajchenbach-Teller
 wrote:
> Why both createFile, open{Read, Write, Append} and get? Is it to avoid a
> signature with dependent types?

I think you are asking why have createFile when it can be implemented
using openWrite or openAppend instead?

We believe that one of the most common operations is simply "store
data X as a new file with filename Y". Having a simple function,
createFile, to accomplish this seems beneficial.

Compare

// With having creatFile:
navigator.getFilesystem().then(function(root) {
  root.createFile("myfile.txt", { data: xhr.response });
});

// Only using openWrite
navigator.getFilesystem().then(function(root) {
  return root.openWrite("myfile.txt");
}).then(function(handle) {
  return Promise.every(
handle.setSize(0),
handle.write(xhr.response));
});

Does that answer the question?

/ Jonas


> Cheers,
>  David
>
> On 7/13/13 2:31 AM, Jonas Sicking wrote:
>>
>>   Promise createFile(DOMString path, MakeFileOptions options);
>>   Promise createDirectory(DOMString path);
>>
>>   Promise<(File or Directory)> get(DOMString path);
>>
>>   Promise move((DOMString or File or Directory) entry,
>>  (DOMString or Directory or DestinationDict) dest);
>>   Promise copy((DOMString or File or Directory) entry,
>>  (DOMString or Directory or DestinationDict) dest);
>>   Promise remove((DOMString or File or Directory) path,
>>optional DeleteMode recursive = "nonrecursive");
>>
>>   Promise openRead((DOMString or File) file);
>>   Promise openWrite((DOMString or File) file,
>> optional CreateMode createMode = "createifneeded");
>>   Promise openAppend((DOMString or File) file,
>> optional CreateMode createMode = "createifneeded");
>>
>>   EventStream<(File or Directory)> enumerate();
>>   EventStream enumerateDeep();
>> };
>
>
> --
> David Rajchenbach-Teller, PhD
>  Performance Team, Mozilla

Polished FileSystem API proposal

[Jonas Sicking]

Hi All,

Yesterday a few of us at mozilla went through the FileSystem API
proposal we previously sent [1] and tightened it up.

Executive Summary (aka TL;DR):
Below is the mozilla proposal for a simplified filesystem API. It
contains two new abstractions, a Directory object which allows
manipulating files and directories within it, and a FileHandle object
which allows holding an exclusive lock on a file while performing
multiple read/write operations on it.

It's largely modeled after posix, but because we've tried to keep it
author friendly despite it's asynchronous nature, it differs in a few
cases.

There are opportunities for further simplifications by straying
further from posix. It's unclear if this is desired or not.

Detailed proposal:

partial interface Navigator {
  Promise getFilesystem(optional FilesystemParameters parameters);
};

interface Directory {
  readonly attribute DOMString name;

  Promise createFile(DOMString path, MakeFileOptions options);
  Promise createDirectory(DOMString path);

  Promise<(File or Directory)> get(DOMString path);

  Promise move((DOMString or File or Directory) entry,
 (DOMString or Directory or DestinationDict) dest);
  Promise copy((DOMString or File or Directory) entry,
 (DOMString or Directory or DestinationDict) dest);
  Promise remove((DOMString or File or Directory) path,
   optional DeleteMode recursive = "nonrecursive");

  Promise openRead((DOMString or File) file);
  Promise openWrite((DOMString or File) file,
optional CreateMode createMode = "createifneeded");
  Promise openAppend((DOMString or File) file,
optional CreateMode createMode = "createifneeded");

  EventStream<(File or Directory)> enumerate();
  EventStream enumerateDeep();
};

interface FileHandle
{
  readonly attribute FileOpenMode mode;
  readonly attribute boolean active;

  attribute long long? location;

  Promise getFile();
  AbortableProgressPromise read(unsigned long long size);
  AbortableProgressPromise readText(unsigned long long
size, optional DOMString encoding = "utf-8");

  void abort();
};

interface FileHandleWritable : FileHandle
{
  AbortableProgressPromise write((DOMString or ArrayBuffer or
ArrayBufferView or Blob) value);

  Promise setSize(optional unsigned long long size);

  Promise flush();
};

partial interface URL {
  static DOMString? getPersistentURL(File file);
}

// WebIDL cruft that's largely transparent
enum PersistenceType { "temporary", "persistent" };
dictionary FilesystemParameters {
  PersistenceType storage = "temporary";
};

dictionary MakeFileOptions {
  boolean overwriteIfExists = false;
  (DOMString or Blob or ArrayBuffer or ArrayBufferView) data;
};

enum CreateMode { "createifneeded", "dontcreate" }
enum DeleteMode { "recursive", "nonrecursive" }

dictionary DestinationDict {
  Directory dir;
  DOMString name;
};

enum FileOpenMode { "read", "write", "append" };

So this API introduces 2 classes: Directory and FileHandle. Directory
allows manipulation of the files and directories stored inside that
directory. FileHandle represents an exclusively opened file and allows
manipulation of the file contents.

The behavior is hopefully mostly obvious. A few general comments:

The functions on Directory that accept DOMString arguments for
filenames allow names like "path/to/file". If the function creates a
file, then it creates the intermediate directories. Such paths are
always interpreted as relative to the directory itself, never relative
to the root.

We were thinking of *not* allowing paths that walk up the directory
tree. So paths like "../foo", "..", "/foo/bar" or "foo/../bar" are not
allowed. This to keep things simple and avoid security issues for the
page.

Likewise, passing a File object to an operation of Directory where the
File object isn't contained in that directory or its descendents also
results in an error.

One thing that is probably not obvious is how the FileHandle.location
attribute works. This attribute is used by the read/readText/write
functions to select where the read or write operation starts. When
.read is called, it uses the current value of .location to determine
where the reading starts. It then fires off an asynchronous read
operation. It finally synchronously increases .location by the amount
of the 'size' argument before returning. Same thing for .write() and
.readText().

This means that the caller can simply set .location and then fire off
multiple read or write operations which automatically will happen
staggered in the file. It also means that the caller can set the
location for next operation by simply setting .location, or can check
the current location by simply getting .location.

Setting .loca

Proposal for a filesystem API

[Jan Varga]

Hi WebApps!

There has been a lot of controversy about implementing a file system API in
other browsers. Firefox and IE already support storing files in IndexedDB,
but it seems that web developers need something simpler in some cases and
there's also the filesystem URL scheme that is not covered by files in
IndexedDB.

This new proposal is heavily based on Apple's minimal filesystem API
proposal [1], on Mozilla's DeviceStorage API 2 [2] and on Mozilla's current
FileHandle API [3]. It is only designed for the use case of sandboxed local
file storage.

In Firefox, the FileHandle API provides the ability to write to a file, as
well as the locking mechanisms needed to allow writing safely. The
FileHandle interface is currently supported in IndexedDB and it is also
planned for DeviceStorage API. However, the new proposal for the file
system API uses a modified version of the FileHandle API. The FileHandle
interface acts exactly like LockedFile in the current FileHandle API. I.e.
it represents an opened file and it automatically closes if you stop
posting requests against it (like transactions in IDB except it doesn't
provide any rollback functionality, i.e. only Isolation of the ACID).

There's a documentation for the current FileHandle API [4], which contains
detailed description of the LockedFile interface (that now becomes the new
FileHandle interface).


Here's the proposed file system API:

Getting the root directory:

enum PersistenceType { "temporary", "permanent" };

dictionary FilesystemParameters
{
  PersistenceType storage;
};

partial interface Navigator {
  Directory getSandboxedFilesystem(optional FilesystemParameters
parameters);
};


The Directory interface:

enum FileOpenMode { "read", "write", "append" };

interface Directory {
  readonly attribute DOMString name;

  // Asynchronously returns boolean true or false to indicate success.
  Future
  create(DOMString name, optional Blob blob);

  // Asynchronously returns the newly created Directory.
  Future
  makeDirectory(DOMString name);

  // Asynchronously returns a File or Directory. If the entry is a File
  // then get() is a shortcut for openForReading().getFile()
  Future
  get(DOMString name);

  // Deletes a file or directory; Asynchronously returns boolean true or
false
  // to indicate success.
  Future
  remove(DOMString name);

  // Atomically renames a file or directory; Asynchronously returns boolean
  // true or false to indicate success.
  Future
  rename(DOMString oldName, DOMString newName);

  // alternative method for rename():

  // Atomically moves or renames a file or directory; Asynchronously returns
  // boolean true or false to indicate success.
  Future
  moveTo((DOMString or File or Directory) entry, DOMString newName,
optional Directory newParentDirectory);

  // TODO: maybe add copyTo() method

  // Asynchronously returns a FileHandle; error if called on a Directory.
  // If a File is passed, it must be a descendant of this directory.
  Future
  openRead((DOMString or File) file);

  // Asynchronously returns a FileHandleWritable;
  // error if called on a Directory.
  // If a File is passed, it must be a descendant of this directory.
  Future
  openWrite((DOMString or File) file);

  // Asynchronously returns a FileHandleWritable;
  // error if called on a Directory.
  // If a File is passed, it must be a descendant of this directory.
  Future
  openAppend((DOMString or File) file);

  // alternative method for openRead(), openWrite(), openAppend():

  Future<(FileHandle or FileHandleWritable)>
  open((DOMString or File) file, optional FileOpenMode mode);

  // TODO: Solve futures way of doing a cursor-style API
  // Asynchronously returns File or Directory objects.
  CursorFuture
  enumerateShallow(); // or readEntries();

  // Asynchronously returns File objects
  CursorFuture
  enumerateDeep(); // or readEntriesRecursively();
};


The FileHandle/FileHandleWritable interface, representing a file open in
read, write or append mode.

dictionary MetadataParameters
{
  boolean size = false;
  boolean lastModified = false;
};

interface FileHandle
{
  // "read" or "write" or "append"
  readonly attribute FileOpenMode mode;

  readonly attribute boolean active;

  // Offset in the file. This value is changed automatically after every
read
  // and every write. Reads and writes always start at the location.
  // null means end-of-file
  attribute any location;

  // Upon success, you receive a read-only "snapshot" of the file's content
in
  // the form of a File instance (that can be used anywhere a Blob is
accepted,
  // like FileReader, XMLHttpRequest, IndexedDB, etc).
  Future getFile();

  // Asynchronously returns an ArrayBuffer of the given size. The operation
  // starts at the location. Moving the location by the number of bytes
read.
  ProgressFuture
  

Re: FileSystem API: Cross origin sharing

[Glenn Maynard]

On Tue, Aug 14, 2012 at 4:51 PM, Charles Pritchard  wrote:

> Is there a long term plan for transferring access to Files from the
> FileSystem?
>
> Currently, we can using FileEntry.file semantics then post the resulting
> Blob to another frame. From that point, the receiver can create a Blob URL.
> I'm concerned about all the extra work with Blobs which may get stuck in
> RAM.
>

How?  The only possible way is object URL leaks (missed revokeObjectURL
calls), which should be addressed by autoRevoke.

-- 
Glenn Maynard

FileSystem API: Cross origin sharing

[Charles Pritchard]

Is there a long term plan for transferring access to Files from the FileSystem?

Currently, we can using FileEntry.file semantics then post the resulting Blob 
to another frame. From that point, the receiver can create a Blob URL. I'm 
concerned about all the extra work with Blobs which may get stuck in RAM.

Sure would be nice to pass strings instead.


-Charles

Re: FileSystem API: Adding file size field to Metadata?

[Eric Uhrhane]

On Tue, Feb 28, 2012 at 1:52 PM, Darin Fisher  wrote:
> On Tue, Feb 28, 2012 at 10:47 AM, Kinuko Yasuda  wrote:
>>
>> Hi,
>>
>> While looking at the FileSystem API draft I noticed that we only expose
>> 'modificationTime' in 'Metadata' object.  Since FileEntry itself doesn't
>> have 'size' field unlike File object maybe it's reasonable to add 'size'
>> field to Metadata?
>>
>> http://www.w3.org/TR/file-system-api/#the-metadata-interface
>>
>> Without adding this we can indirectly get the file size by creating 'File'
>> object via FileEntry.file() method and accessing File.size, but when an app
>> wants to display the modificationTime and file size at once (and it sounds
>> very plausible use case) the app needs to call two different async methods--
>> which doesn't sound very nice.  WDYT?
>>
>> Thanks,
>> Kinuko
>>
>>
>
>
> I think this is a nice improvement.  File size is very obviously something
> one might expect to be included in meta data for a file :-)
>
> -Darin

Yup.  I've had this recommended offline at Google as a way to speed up
directory listings.  Currently, for each Entry that you get from a
DirectoryReader, you have to call getMetadata to get mtime, then
getFile to get size.  This cuts the number of calls in half, which can
be a pretty big deal for a large directory.  I'll put it in.

Charles, I'd like to leave the may-be-nice ones out for now--it's easy
to add them later, and the current design also makes it easy for
implementers to add other fields experimentally.  If others think that
this rises to the same importance of size, we can certainly discuss
it.

Re: FileSystem API: Adding file size field to Metadata?

[Charles Pritchard]

On Feb 28, 2012, at 1:52 PM, Darin Fisher  wrote:

> On Tue, Feb 28, 2012 at 10:47 AM, Kinuko Yasuda  wrote:
> Hi,
> 
> While looking at the FileSystem API draft I noticed that we only expose 
> 'modificationTime' in 'Metadata' object.  Since FileEntry itself doesn't have 
> 'size' field unlike File object maybe it's reasonable to add 'size' field to 
> Metadata?
> 
> http://www.w3.org/TR/file-system-api/#the-metadata-interface
> 
> Without adding this we can indirectly get the file size by creating 'File' 
> object via FileEntry.file() method and accessing File.size, but when an app 
> wants to display the modificationTime and file size at once (and it sounds 
> very plausible use case) the app needs to call two different async methods-- 
> which doesn't sound very nice.  WDYT?
> 
> Thanks,
> Kinuko
> 
> 
> 
> 
> I think this is a nice improvement.  File size is very obviously something 
> one might expect to be included in meta data for a file :-)

Content type may be nice as well. Whatever the OS reports.

-Charles

Re: FileSystem API: Adding file size field to Metadata?

[Darin Fisher]

On Tue, Feb 28, 2012 at 10:47 AM, Kinuko Yasuda  wrote:

> Hi,
>
> While looking at the FileSystem API draft I noticed that we only expose
> 'modificationTime' in 'Metadata' object.  Since FileEntry itself doesn't
> have 'size' field unlike File object maybe it's reasonable to add 'size'
> field to Metadata?
>
> http://www.w3.org/TR/file-system-api/#the-metadata-interface
>
> Without adding this we can indirectly get the file size by creating 'File'
> object via FileEntry.file() method and accessing File.size, but when an app
> wants to display the modificationTime and file size at once (and it sounds
> very plausible use case) the app needs to call two different async
> methods-- which doesn't sound very nice.  WDYT?
>
> Thanks,
> Kinuko
>
>
>

I think this is a nice improvement.  File size is very obviously something
one might expect to be included in meta data for a file :-)

-Darin

FileSystem API: Adding file size field to Metadata?

[Kinuko Yasuda]

Hi,

While looking at the FileSystem API draft I noticed that we only expose
'modificationTime' in 'Metadata' object.  Since FileEntry itself doesn't
have 'size' field unlike File object maybe it's reasonable to add 'size'
field to Metadata?

http://www.w3.org/TR/file-system-api/#the-metadata-interface

Without adding this we can indirectly get the file size by creating 'File'
object via FileEntry.file() method and accessing File.size, but when an app
wants to display the modificationTime and file size at once (and it sounds
very plausible use case) the app needs to call two different async
methods-- which doesn't sound very nice.  WDYT?

Thanks,
Kinuko

Re: FileSystem API - The Flags interface

[Eric U]

The exception is thrown by getFile on DirectoryEntrySync, not by the
Flags constructor; both the example and the flags interface are
correct.

On Sat, Oct 8, 2011 at 11:54 AM, Bronislav Klučka
 wrote:
> Hello,
> http://www.w3.org/TR/file-system-api/#the-flags-interface
> If you look at the description of exclusive flag (4.2.1), the description
> states "no exception", but the example (4.2.2) uses exception to determine
> whether file already existed.
> So the question is, what is wrong: the description or example?
>
> Brona Klucka
>
>
>

FileSystem API - The Flags interface

[Bronislav Klučka]

Hello,
http://www.w3.org/TR/file-system-api/#the-flags-interface
If you look at the description of exclusive flag (4.2.1), the 
description states "no exception", but the example (4.2.2) uses 
exception to determine whether file already existed.

So the question is, what is wrong: the description or example?

Brona Klucka

Re: HTML5 Filesystem API feedback

[Robert Ginda]

On Mon, Apr 18, 2011 at 5:28 PM, Eric Uhrhane  wrote:
> On Fri, Apr 15, 2011 at 4:01 PM, Arun Ranganathan  wrote:
>> On 4/15/11 6:29 PM, Aryeh Gregor wrote:
>>>
>>> On Wed, Apr 13, 2011 at 4:35 PM, Robert Ginda  wrote:

 * The FileError object is a bit awkward to work with.  I found that I
 frequently had every reason to expect my calls to succeed (because
 they were a follow-on to something that already succeeded), but I
 wanted to log the failure reason in the event they didn't.  The code
 online suggests a switch/case statement to turn error codes into
 mnemonic strings.  This requires a hardcoded list of all known errors,
 and a call out to this utility function every time you want to display
 the error reason in a readable way.  I suggest adding the mnemonic
 string as a property of FileError, and displaying it as part of the
 toString.  (See util.getFileErrorMnemonic() and
 util.installFileErrorToString() for example implementations.)
>>>
>>> I'd suggest solving this the same way as DOMException does, for
>>> consistency:
>>>
>>>
>>> http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#exception-domexception
>>>
>>> It has a "name" property that contains the name, e.g.,
>>> "INDEX_SIZE_ERR" or "INVALID_STATE_ERR".  Of course, if you want to
>>> display a real error message you have to localize it, but an
>>> easily-recognizable, memorable, Google-able string is still extremely
>>> useful in many cases.  Plus, DOMException already does it and it's
>>> about two lines to spec, so why not?
>>>
>>
>> This sounds like a reasonable addition to both FileException and FileError;
>> I'll make this change in File API.
>
> That sounds great.
>

Yes, sounds good to me too.  Thanks Arun.


Rob.

Re: HTML5 Filesystem API feedback

[Eric Uhrhane]

On Fri, Apr 15, 2011 at 4:01 PM, Arun Ranganathan  wrote:
> On 4/15/11 6:29 PM, Aryeh Gregor wrote:
>>
>> On Wed, Apr 13, 2011 at 4:35 PM, Robert Ginda  wrote:
>>>
>>> * The FileError object is a bit awkward to work with.  I found that I
>>> frequently had every reason to expect my calls to succeed (because
>>> they were a follow-on to something that already succeeded), but I
>>> wanted to log the failure reason in the event they didn't.  The code
>>> online suggests a switch/case statement to turn error codes into
>>> mnemonic strings.  This requires a hardcoded list of all known errors,
>>> and a call out to this utility function every time you want to display
>>> the error reason in a readable way.  I suggest adding the mnemonic
>>> string as a property of FileError, and displaying it as part of the
>>> toString.  (See util.getFileErrorMnemonic() and
>>> util.installFileErrorToString() for example implementations.)
>>
>> I'd suggest solving this the same way as DOMException does, for
>> consistency:
>>
>>
>> http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#exception-domexception
>>
>> It has a "name" property that contains the name, e.g.,
>> "INDEX_SIZE_ERR" or "INVALID_STATE_ERR".  Of course, if you want to
>> display a real error message you have to localize it, but an
>> easily-recognizable, memorable, Google-able string is still extremely
>> useful in many cases.  Plus, DOMException already does it and it's
>> about two lines to spec, so why not?
>>
>
> This sounds like a reasonable addition to both FileException and FileError;
> I'll make this change in File API.

That sounds great.

Re: HTML5 Filesystem API feedback

[Arun Ranganathan]

On 4/15/11 6:29 PM, Aryeh Gregor wrote:

On Wed, Apr 13, 2011 at 4:35 PM, Robert Ginda  wrote:

* The FileError object is a bit awkward to work with.  I found that I
frequently had every reason to expect my calls to succeed (because
they were a follow-on to something that already succeeded), but I
wanted to log the failure reason in the event they didn't.  The code
online suggests a switch/case statement to turn error codes into
mnemonic strings.  This requires a hardcoded list of all known errors,
and a call out to this utility function every time you want to display
the error reason in a readable way.  I suggest adding the mnemonic
string as a property of FileError, and displaying it as part of the
toString.  (See util.getFileErrorMnemonic() and
util.installFileErrorToString() for example implementations.)

I'd suggest solving this the same way as DOMException does, for consistency:

http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#exception-domexception

It has a "name" property that contains the name, e.g.,
"INDEX_SIZE_ERR" or "INVALID_STATE_ERR".  Of course, if you want to
display a real error message you have to localize it, but an
easily-recognizable, memorable, Google-able string is still extremely
useful in many cases.  Plus, DOMException already does it and it's
about two lines to spec, so why not?



This sounds like a reasonable addition to both FileException and 
FileError; I'll make this change in File API.


-- A*

Re: HTML5 Filesystem API feedback

[Aryeh Gregor]

On Wed, Apr 13, 2011 at 4:35 PM, Robert Ginda  wrote:
> * The FileError object is a bit awkward to work with.  I found that I
> frequently had every reason to expect my calls to succeed (because
> they were a follow-on to something that already succeeded), but I
> wanted to log the failure reason in the event they didn't.  The code
> online suggests a switch/case statement to turn error codes into
> mnemonic strings.  This requires a hardcoded list of all known errors,
> and a call out to this utility function every time you want to display
> the error reason in a readable way.  I suggest adding the mnemonic
> string as a property of FileError, and displaying it as part of the
> toString.  (See util.getFileErrorMnemonic() and
> util.installFileErrorToString() for example implementations.)

I'd suggest solving this the same way as DOMException does, for consistency:

http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#exception-domexception

It has a "name" property that contains the name, e.g.,
"INDEX_SIZE_ERR" or "INVALID_STATE_ERR".  Of course, if you want to
display a real error message you have to localize it, but an
easily-recognizable, memorable, Google-able string is still extremely
useful in many cases.  Plus, DOMException already does it and it's
about two lines to spec, so why not?

Re: HTML5 Filesystem API feedback

[Robert Ginda]

Hi Eric,

Thanks for your reply, my comments are inline...

On Fri, Apr 15, 2011 at 11:53 AM, Eric Uhrhane  wrote:
> Robert:
>
> First of all, thanks for the feedback.
>
> On Wed, Apr 13, 2011 at 1:35 PM, Robert Ginda  wrote:
>> Hello public-webapps,
>>
>> I've spent a bit of time with the filesystem API recently, building out a
>> set of common file dialogs (open, save-as, etc) for ChromeOS.  We have
>> a private API call to get access to a special filesystem that contains the
>> user's downloads folder and the mounted external storage, but beyond
>> that the dialogs use the standard filesystem API calls.
>>
>> The bulk of the code is here:
>> http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/resources/file_manager/js/
>>
>> I've run into a few awkward bits that I'd like to bring up here to see what
>> everyone else thinks.
>>
>> * The async API is really awkward when you want to resolve multiple
>> entries.  I have a case where I'd like to resolve a list of
>> directories and then do something when that completes.  It would be
>> helpful if the DirectoryEntry class had a getDirectories() method.
>> I've built my own in file_manager/js/util.js in the function called
>> util.getDirectories.
>
> In general, I'm pretty happy to see this sort of thing.  You may not
> like to hear this, but any problem that can just be solved by layering
> a small javascript library on top of this API is a problem we don't
> need to fix in the first pass of the API.  We want it to start out
> powerful and simple, and eventually, if that library is something that
> absolutely everyone seems to need, we may want to incorporate it.  So
> it is very helpful to see the kinds of libraries that get built on
> top, and thanks for sending the pointer, but I suggest we wait and see
> before making any changes.
>

Sure, I see your point.  I just think this is such an obvious problem
(introduced by the necessarily asynchronous nature of the API), that
it would be nice to offer a solution up front.  Anything that isn't in
the first pass of the API is likely to stick around in shim libraries
for a long long time.

> BTW, you can make your getDirectories method faster if you don't wait
> for a response before sending off the next query [assuming no memory
> constraints].  Just fire off all the requests in one go, then keep a
> countdown of responses in your callback-wrapper.  When your countdown
> hits zero, fire the "we're done" signal.
>

Yeah, I've actually used the countdown version of this pattern in
another part of the code.  I probably should have used it here too.
FWIW, it gets a bit difficult if you care about getting called back in
the correct order.

>> * That leads to a desire to get "entries" in the general sense.  In my
>> case I knew ahead of time that I had a list of directories.  If I
>> didn't know whether I had been handed directories or files, the code
>> would get pretty hairy.  So in addition to getDirectories, a
>> getEntries method (and getEntry, too) would be super useful.
>
> This is a method that would look up, but not create, an Entry, given a
> path?  Can you outline the circumstances in which you'd use this, but
> where you wouldn't want to list the whole parent directory?
>

Say you wanted to implement the shell builtin 'mv  ...
'.  You can have an arbitrary number of source entries, each
of which can be either a file or a directory, and can be located at an
arbitrary depth in the filesystem.  The , as well, could be an
existing file or directory.

With the existing API, you would have to try each source first as one
type (say, getFile), then if that fails (with a TYPE_MISMATCH_ERR),
try again using the other getter.  A generic getEntry() would make
this considerably cleaner.

>> * Another helper I found useful was util.forEachDirEntry from the same
>> file previously mentioned.  This function invokes a callback once for
>> each entry in a directory.  This saves the caller from having to
>> create the reader and walk the results array during the callback.  It
>> makes client code significantly more readable.  I'd love to see
>> DirectoryEntry.forEach(...).
>
> I can see the utility there pretty clearly.
>
>> * The FileError object is a bit awkward to work with.  I found that I
>> frequently had every reason to expect my calls to succeed (because
>> they were a follow-on to something that already succeeded), but I
>> wanted to log the failure reason in the event they didn't.  The code
>> online suggests a switch/case statement to turn error codes into
>> mnemonic strings.  This 

Re: HTML5 Filesystem API feedback

[Eric Uhrhane]

Robert:

First of all, thanks for the feedback.

On Wed, Apr 13, 2011 at 1:35 PM, Robert Ginda  wrote:
> Hello public-webapps,
>
> I've spent a bit of time with the filesystem API recently, building out a
> set of common file dialogs (open, save-as, etc) for ChromeOS.  We have
> a private API call to get access to a special filesystem that contains the
> user's downloads folder and the mounted external storage, but beyond
> that the dialogs use the standard filesystem API calls.
>
> The bulk of the code is here:
> http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/resources/file_manager/js/
>
> I've run into a few awkward bits that I'd like to bring up here to see what
> everyone else thinks.
>
> * The async API is really awkward when you want to resolve multiple
> entries.  I have a case where I'd like to resolve a list of
> directories and then do something when that completes.  It would be
> helpful if the DirectoryEntry class had a getDirectories() method.
> I've built my own in file_manager/js/util.js in the function called
> util.getDirectories.

In general, I'm pretty happy to see this sort of thing.  You may not
like to hear this, but any problem that can just be solved by layering
a small javascript library on top of this API is a problem we don't
need to fix in the first pass of the API.  We want it to start out
powerful and simple, and eventually, if that library is something that
absolutely everyone seems to need, we may want to incorporate it.  So
it is very helpful to see the kinds of libraries that get built on
top, and thanks for sending the pointer, but I suggest we wait and see
before making any changes.

BTW, you can make your getDirectories method faster if you don't wait
for a response before sending off the next query [assuming no memory
constraints].  Just fire off all the requests in one go, then keep a
countdown of responses in your callback-wrapper.  When your countdown
hits zero, fire the "we're done" signal.

> * That leads to a desire to get "entries" in the general sense.  In my
> case I knew ahead of time that I had a list of directories.  If I
> didn't know whether I had been handed directories or files, the code
> would get pretty hairy.  So in addition to getDirectories, a
> getEntries method (and getEntry, too) would be super useful.

This is a method that would look up, but not create, an Entry, given a
path?  Can you outline the circumstances in which you'd use this, but
where you wouldn't want to list the whole parent directory?

> * Another helper I found useful was util.forEachDirEntry from the same
> file previously mentioned.  This function invokes a callback once for
> each entry in a directory.  This saves the caller from having to
> create the reader and walk the results array during the callback.  It
> makes client code significantly more readable.  I'd love to see
> DirectoryEntry.forEach(...).

I can see the utility there pretty clearly.

> * The FileError object is a bit awkward to work with.  I found that I
> frequently had every reason to expect my calls to succeed (because
> they were a follow-on to something that already succeeded), but I
> wanted to log the failure reason in the event they didn't.  The code
> online suggests a switch/case statement to turn error codes into
> mnemonic strings.  This requires a hardcoded list of all known errors,
> and a call out to this utility function every time you want to display
> the error reason in a readable way.  I suggest adding the mnemonic
> string as a property of FileError, and displaying it as part of the
> toString.  (See util.getFileErrorMnemonic() and
> util.installFileErrorToString() for example implementations.)

Hmm...I'm not sure what other APIs usually do about this.  As soon as
you start adding strings that you intend to be human-readable, you
start getting into all sorts of internationalization issues.  Anyone?

Eric

HTML5 Filesystem API feedback

[Robert Ginda]

Hello public-webapps,

I've spent a bit of time with the filesystem API recently, building out a
set of common file dialogs (open, save-as, etc) for ChromeOS.  We have
a private API call to get access to a special filesystem that contains the
user's downloads folder and the mounted external storage, but beyond
that the dialogs use the standard filesystem API calls.

The bulk of the code is here:
http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/resources/file_manager/js/

I've run into a few awkward bits that I'd like to bring up here to see what
everyone else thinks.

* The async API is really awkward when you want to resolve multiple
entries.  I have a case where I'd like to resolve a list of
directories and then do something when that completes.  It would be
helpful if the DirectoryEntry class had a getDirectories() method.
I've built my own in file_manager/js/util.js in the function called
util.getDirectories.

* That leads to a desire to get "entries" in the general sense.  In my
case I knew ahead of time that I had a list of directories.  If I
didn't know whether I had been handed directories or files, the code
would get pretty hairy.  So in addition to getDirectories, a
getEntries method (and getEntry, too) would be super useful.

* Another helper I found useful was util.forEachDirEntry from the same
file previously mentioned.  This function invokes a callback once for
each entry in a directory.  This saves the caller from having to
create the reader and walk the results array during the callback.  It
makes client code significantly more readable.  I'd love to see
DirectoryEntry.forEach(...).

* The FileError object is a bit awkward to work with.  I found that I
frequently had every reason to expect my calls to succeed (because
they were a follow-on to something that already succeeded), but I
wanted to log the failure reason in the event they didn't.  The code
online suggests a switch/case statement to turn error codes into
mnemonic strings.  This requires a hardcoded list of all known errors,
and a call out to this utility function every time you want to display
the error reason in a readable way.  I suggest adding the mnemonic
string as a property of FileError, and displaying it as part of the
toString.  (See util.getFileErrorMnemonic() and
util.installFileErrorToString() for example implementations.)


Rob.

Re: [whatwg] Intent of the FileSystem API

[Glenn Maynard]

On Tue, Mar 1, 2011 at 3:17 PM, Eric Uhrhane  wrote:
>> Maybe disallow nulls; I'm not sure if this is special enough to
>> actually need to be in this set.
>
> I like it, though, as allowing nulls in strings is likely to lead to user 
> error.

In my experience, most of the errors surrounding nulls are related to
null-terminated strings, which doesn't apply in JS.

I'm not bothered much either way, though: no real systems support
nulls in filenames (since it would break just about all C APIs), so
disallowing it won't break anything.

-- 
Glenn Maynard

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

On Tue, Mar 1, 2011 at 11:37 AM, Glenn Maynard  wrote:
> On Tue, Mar 1, 2011 at 1:13 PM, Eric Uhrhane  wrote:
>> What would you suggest for limitations?  If we're requiring
>> virtualization, it seems to me that we could be quite liberal.
>
> I'd suggest only the restrictions that are required for the API: no
> "", ".", "..", and no filenames containing forward slashes.

That makes sense.

> Maybe some fairly high filename component limit (NAME_MAX) for sanity;
> maybe 4k (in UTF-16).
>
> Maybe disallow nulls; I'm not sure if this is special enough to
> actually need to be in this set.

I like it, though, as allowing nulls in strings is likely to lead to user error.

> I think the "5000 entries per directory" limitation should also go
> away.  Virtualization hides any local directory
> limitations--presumably files on disk wouldn't mirror the virtualized
> directory structure.

Right.

> (By the way, I'm not sure from the spec which error code is used for
> invalid filenames.  Also--though this section may be irrelevant if you
> go with this approach--section 8 says "This section is non-normative."
> and then contains normative requirements.)

The non-normative flag is only supposed to be for the next paragraph,
not the whole section.  Thanks--I'll fix that when I rewrite that
section.

>> Yes, and I'm finding myself agreeing with you, although I've argued
>> the other side in the past.  Sometimes you just have to try
>> implementing it to figure out what's not going to work.
>
> Sure, I don't think anyone predicted OS-related problems that were
> quite that difficult to mask with filename restrictions.  (It feels
> like I should have thought of it, too, since I've hit the this issue
> several times natively when extracting archives with long
> filenames...)
>
> --
> Glenn Maynard
>

Re: [whatwg] Intent of the FileSystem API

[Glenn Maynard]

On Tue, Mar 1, 2011 at 1:13 PM, Eric Uhrhane  wrote:
> What would you suggest for limitations?  If we're requiring
> virtualization, it seems to me that we could be quite liberal.

I'd suggest only the restrictions that are required for the API: no
"", ".", "..", and no filenames containing forward slashes.

Maybe some fairly high filename component limit (NAME_MAX) for sanity;
maybe 4k (in UTF-16).

Maybe disallow nulls; I'm not sure if this is special enough to
actually need to be in this set.

I think the "5000 entries per directory" limitation should also go
away.  Virtualization hides any local directory
limitations--presumably files on disk wouldn't mirror the virtualized
directory structure.

(By the way, I'm not sure from the spec which error code is used for
invalid filenames.  Also--though this section may be irrelevant if you
go with this approach--section 8 says "This section is non-normative."
and then contains normative requirements.)

> Yes, and I'm finding myself agreeing with you, although I've argued
> the other side in the past.  Sometimes you just have to try
> implementing it to figure out what's not going to work.

Sure, I don't think anyone predicted OS-related problems that were
quite that difficult to mask with filename restrictions.  (It feels
like I should have thought of it, too, since I've hit the this issue
several times natively when extracting archives with long
filenames...)

-- 
Glenn Maynard

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

On Mon, Feb 28, 2011 at 6:21 PM, Glenn Maynard  wrote:
> On Mon, Feb 28, 2011 at 7:41 PM, Eric Uhrhane  wrote:
>>
>> Sorry--I meant to push this over to public-webapps, as Ian suggested.
>> [+cc public-webapps, whatwg->BCC]
>
> Currently (reviewing for context), the spec tries to reach "filename
> consistency" (only allowing filenames to be used which can be used
> consistently on all browsers) by restricting filenames to the lowest-common
> denominator.  With this path length issue, I don't think that approach is
> correct, since it seems like everyone's going to have to implement path
> virtualization anyway.

Indeed.

> Based on that, I'd like to suggest a model change:
>
> - Only place basic restrictions on filenames, giving the broadest set of
> filenames that are generally supported (eg. no slashes, nulls, and some
> reasonable length limits).

What would you suggest for limitations?  If we're requiring
virtualization, it seems to me that we could be quite liberal.

> - For sandboxed directories, it's guaranteed that all legal filenames are
> permitted, whether or not the local filesystem supports them.  If the native
> filesystem isn't capable of supporting all legal filenames, virtualization
> is required.
> - For "mounted" directories, no guarantee of "filename consistency" is made:
> you can create filenames on some systems that aren't allowed on others.
> Filenames invalid on a particular system raise an exception.
>
> This gives filename consistency for sandboxed directories without applying
> quirks of all operating systems to users of all other operating systems.
>
> This expressly gives no guarantee of filename consistency for "mounted"
> directories (which don't exist yet, of course).  As we talked about before,
> I think this is critical when dealing with directories shared with native
> applications.  If the user creates "hello?.txt", then a web app version of
> gzip must be able to create "hello?.txt.gz" if the system can do so.
> Similarly, if the user creates "family photos: Bill.jpg", a web app image
> resizer must be able to create "family photos: Bill (thumbnail).jpg".
>
> (I'm not trying to get into the particulars of "mounted" directories now,
> but I needed to mention them to explain how this suggestion fits together,
> since directories interacting with native applications can't be
> virtualized.)

Yes, and I'm finding myself agreeing with you, although I've argued
the other side in the past.  Sometimes you just have to try
implementing it to figure out what's not going to work.

> On Mon, Feb 28, 2011 at 7:59 PM, Charles Pritchard  wrote:
>> There are certainly issues in old copies of windows, but I think they've
>> been addressed since.
>
> It still happens in Win7, even in Windows Explorer.  I'm pretty sure this is
> just PATH_MAX, which hasn't changed.
>
> --
> Glenn Maynard
>
>

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

On Mon, Feb 28, 2011 at 4:59 PM, Charles Pritchard  wrote:
> On 2/28/2011 4:10 PM, Eric Uhrhane wrote:
>>
>> On Mon, Feb 28, 2011 at 2:54 PM, Charles Pritchard
>>  wrote:
>>>
>>> I'd like some clarification on the intent of the FileSystem API:
>>> requestFileSystem permanent, getDirectory and getFile.
>
> ...
>>>
>>> They're not particularly easy to use, compared to the IndexedDB data
>>> store,
>>
>> I find files to be easier to use than databases, but I'm clearly
>> biased ;'>.  However, that's not really the point.  Files and
>> databases serve different needs, and various use cases are better
>> served by each than the other.  If you want transactions or indexing,
>> you want IDB.  If you want fast, mutable binary blobs with persistent
>> URLs, you want FileSystem.
>
> It's much easier to send a query via IndexedDB to "list all files" in a
> directory
> mask, than it is to build up the same thing using getDirectory.
>
> I'm quite happy with File, FileWriter, and so forth.
>
> It's much easier/quicker to post a binary blob at an arbitrary path using a
> data store, than it is going through full file system
> semantics; a bunch of create directory and file calls.
>
> For scalability, you've got Amazon S3 and Google Storage as data store
> models.
>
> I target the old WebSQL, localStorage, file system,
> IDB and remote services, to stash images and metadata.
>
> IDB has been the easiest to support, as the others generally require more
> work to stash metadata information,
> and have uncertain locking.
>
>
>>> but if they are OS level commands,
>>> then they do help with usability of a web app with the rest of the OS:
>>> with
>>> managing files directly from the OS file manager.
>>>
>>> Is the FileSystem API intended to expose a folder on the host operating
>>> system, or is it just an abstraction
>>> which can be bolted onto any data system (such as, using SQLite as a
>>> backend, as appCache and IndexedDB do)  ?
>>
>> While the current API could in theory be implemented either way, there
>> have been many requests for expansions to the API that would allow for
>> access to directories outside the sandbox, e.g. giving Flickr access
>> to "My Photos".  While that's not yet been specced, that would clearly
>> require real pass-through access and real file names.  The rest of the
>> API could of course be identical, and I hope that eventually it will
>> serve both use cases.
>
> I'm supportive of a virtual FS for the temporary file system, but
> using a real sandbox for the permanent system.

I don't think it's likely that we'll want to have different APIs for
temporary vs. persistent storage, in the origin-specific sandbox.  But
if you mean a different API [different path limitations, anyway] for
the sandbox vs. other "mount points" such as "My Photos", then yes, I
think that makes sense.

> I'm sure we'll get there eventually, serving both cases.
>
>>> Are there any plans of reviewing/visiting efficient File/Blob support in
>>> indexeddb?
>>
>> I can't speak for the indexedDB developers, but the last I heard
>> on-list, they were at least hoping to support immutable Blobs, but not
>> soon.
>
> What's the hold up? Working with base64 encoding is slow.
> createObjectURL is nice.
>
>> More on those portability issues:
>>
>> We've been implementing the current FileSystem API spec in Chrome, and
>> had planned that our implementation of the sandbox would be just a
>> passthrough to a directory under the user's profile.  We've been
>> testing that out, currently only allowing access to extensions and
>> apps, but we've gotten feedback that long profile paths are causing a
>> problem on Windows.
>
> Yes, I submitted that feedback :-)
>
> My understanding is that it's quite reasonable to get longer paths than the
> current 255 bytes.
> Keep in mind, that includes the whole path to the AppData profile directory.
>
> There are certainly issues in old copies of windows, but I think they've
> been addressed since.
>
>> speed of access and ease of modification, but we'll be storing the
>> paths in an internal database.
>
> I did the same on the file system as well, as I needed that flexibility
> to get around the various constraints.
>
> I kept original file names, which caused some problems with the 255 byte
> limit.
>
> stashed the file in
> sha1hash.subst

Re: [whatwg] Intent of the FileSystem API

[Glenn Maynard]

On Mon, Feb 28, 2011 at 7:41 PM, Eric Uhrhane  wrote:

> Sorry--I meant to push this over to public-webapps, as Ian suggested.
> [+cc public-webapps, whatwg->BCC]
>

Currently (reviewing for context), the spec tries to reach "filename
consistency" (only allowing filenames to be used which can be used
consistently on all browsers) by restricting filenames to the lowest-common
denominator.  With this path length issue, I don't think that approach is
correct, since it seems like everyone's going to have to implement path
virtualization anyway.

Based on that, I'd like to suggest a model change:

- Only place basic restrictions on filenames, giving the broadest set of
filenames that are generally supported (eg. no slashes, nulls, and some
reasonable length limits).
- For sandboxed directories, it's guaranteed that all legal filenames are
permitted, whether or not the local filesystem supports them.  If the native
filesystem isn't capable of supporting all legal filenames, virtualization
is required.
- For "mounted" directories, no guarantee of "filename consistency" is made:
you can create filenames on some systems that aren't allowed on others.
Filenames invalid on a particular system raise an exception.

This gives filename consistency for sandboxed directories without applying
quirks of all operating systems to users of all other operating systems.

This expressly gives no guarantee of filename consistency for "mounted"
directories (which don't exist yet, of course).  As we talked about before,
I think this is critical when dealing with directories shared with native
applications.  If the user creates "hello?.txt", then a web app version of
gzip must be able to create "hello?.txt.gz" if the system can do so.
Similarly, if the user creates "family photos: Bill.jpg", a web app image
resizer must be able to create "family photos: Bill (thumbnail).jpg".

(I'm not trying to get into the particulars of "mounted" directories now,
but I needed to mention them to explain how this suggestion fits together,
since directories interacting with native applications can't be
virtualized.)


On Mon, Feb 28, 2011 at 7:59 PM, Charles Pritchard  wrote:
> There are certainly issues in old copies of windows, but I think they've
> been addressed since.

It still happens in Win7, even in Windows Explorer.  I'm pretty sure this is
just PATH_MAX, which hasn't changed.

-- 
Glenn Maynard

Re: [whatwg] Intent of the FileSystem API

[Charles Pritchard]

On 2/28/2011 4:10 PM, Eric Uhrhane wrote:

On Mon, Feb 28, 2011 at 2:54 PM, Charles Pritchard  wrote:

I'd like some clarification on the intent of the FileSystem API:
requestFileSystem permanent, getDirectory and getFile.

...

They're not particularly easy to use, compared to the IndexedDB data store,

I find files to be easier to use than databases, but I'm clearly
biased ;'>.  However, that's not really the point.  Files and
databases serve different needs, and various use cases are better
served by each than the other.  If you want transactions or indexing,
you want IDB.  If you want fast, mutable binary blobs with persistent
URLs, you want FileSystem.


It's much easier to send a query via IndexedDB to "list all files" in a 
directory

mask, than it is to build up the same thing using getDirectory.

I'm quite happy with File, FileWriter, and so forth.

It's much easier/quicker to post a binary blob at an arbitrary path 
using a data store, than it is going through full file system

semantics; a bunch of create directory and file calls.

For scalability, you've got Amazon S3 and Google Storage as data store 
models.


I target the old WebSQL, localStorage, file system,
IDB and remote services, to stash images and metadata.

IDB has been the easiest to support, as the others generally require 
more work to stash metadata information,

and have uncertain locking.



but if they are OS level commands,
then they do help with usability of a web app with the rest of the OS: with
managing files directly from the OS file manager.

Is the FileSystem API intended to expose a folder on the host operating
system, or is it just an abstraction
which can be bolted onto any data system (such as, using SQLite as a
backend, as appCache and IndexedDB do)  ?

While the current API could in theory be implemented either way, there
have been many requests for expansions to the API that would allow for
access to directories outside the sandbox, e.g. giving Flickr access
to "My Photos".  While that's not yet been specced, that would clearly
require real pass-through access and real file names.  The rest of the
API could of course be identical, and I hope that eventually it will
serve both use cases.

I'm supportive of a virtual FS for the temporary file system, but
using a real sandbox for the permanent system.

I'm sure we'll get there eventually, serving both cases.


Are there any plans of reviewing/visiting efficient File/Blob support in
indexeddb?

I can't speak for the indexedDB developers, but the last I heard
on-list, they were at least hoping to support immutable Blobs, but not
soon.

What's the hold up? Working with base64 encoding is slow.
createObjectURL is nice.


More on those portability issues:

We've been implementing the current FileSystem API spec in Chrome, and
had planned that our implementation of the sandbox would be just a
passthrough to a directory under the user's profile.  We've been
testing that out, currently only allowing access to extensions and
apps, but we've gotten feedback that long profile paths are causing a
problem on Windows.

Yes, I submitted that feedback :-)

My understanding is that it's quite reasonable to get longer paths than 
the current 255 bytes.

Keep in mind, that includes the whole path to the AppData profile directory.

There are certainly issues in old copies of windows, but I think they've
been addressed since.


speed of access and ease of modification, but we'll be storing the
paths in an internal database.

I did the same on the file system as well, as I needed that flexibility
to get around the various constraints.

I kept original file names, which caused some problems with the 255 byte 
limit.


stashed the file in
sha1hash.substr(-2) + '/' + sha1Hash + '/' originalFilename
with metadata in
sha1hash.substr(-2) + '/' + sha1Hash + '/' metaDataHash + '/entry.txt'

The metadata entry.txt files are in there as a precaution and for 
transparency

if the user is accessing things outside of the app.

..

Would your virtual FS be up to including a "Dropbox" style directory,
where users could copy files into, and they'd be accessible to the FS api?

That's also how I modeled mine, including a Dropbox directory underneath
it all, for users to easily copy files into.

In some cases, it's much easier to drag/drop a hundred files into the 
directory,

through the system file manager. Also allows for the OS to treat the dropbox
as a queue: the browser doesn't need to be running to add items to the 
queue.





This then brings up the question of whether one can specify a uniform,
useful interface for the API that *doesn't* require virtualization of
the filesystem, but I'll be bringing that up in another thread.

Well it sounds like there's definitely a case for "mounting" an existing 
FS/directory,
and there's certainly a different case for using a virtual FS for 
efficiency.



-Charles

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

Sorry--I meant to push this over to public-webapps, as Ian suggested.
[+cc public-webapps, whatwg->BCC]

On Mon, Feb 28, 2011 at 4:10 PM, Eric Uhrhane  wrote:
> On Mon, Feb 28, 2011 at 2:54 PM, Charles Pritchard  wrote:
>> I'd like some clarification on the intent of the FileSystem API:
>> requestFileSystem permanent, getDirectory and getFile.
>
> The FileSystem API has several goals.  Some of them are targeted by
> the current spec, and some have been discussed, but put off until
> later.  We didn't want to take on too much in a single pass [hence the
> three layers of file-related specs so far].
>
>> Are they intended to directly pass through to the host operating system, or
>> are they just programming constructs?
>
> The intent for the local sandbox described in the current FileSystem
> spec was that that would be up to the implementer.  If a UA implements
> it as a passthrough, the files could then be accessible to client-side
> apps as well as the web.  That's reason for the restrictions in the
> "uniformity of interface" section--if you're going to expose the true
> local filesystem, you run into different restrictions on names and
> paths on each platform, and we want code that uses this API to run
> everywhere.  However, we've been running into some issues in this area
> with our implementation--more on that below.
>
>> They're not particularly easy to use, compared to the IndexedDB data store,
>
> I find files to be easier to use than databases, but I'm clearly
> biased ;'>.  However, that's not really the point.  Files and
> databases serve different needs, and various use cases are better
> served by each than the other.  If you want transactions or indexing,
> you want IDB.  If you want fast, mutable binary blobs with persistent
> URLs, you want FileSystem.
>
>> but if they are OS level commands,
>> then they do help with usability of a web app with the rest of the OS: with
>> managing files directly from the OS file manager.
>>
>> Is the FileSystem API intended to expose a folder on the host operating
>> system, or is it just an abstraction
>> which can be bolted onto any data system (such as, using SQLite as a
>> backend, as appCache and IndexedDB do)  ?
>
> While the current API could in theory be implemented either way, there
> have been many requests for expansions to the API that would allow for
> access to directories outside the sandbox, e.g. giving Flickr access
> to "My Photos".  While that's not yet been specced, that would clearly
> require real pass-through access and real file names.  The rest of the
> API could of course be identical, and I hope that eventually it will
> serve both use cases.
>
>> Are there any plans of reviewing/visiting efficient File/Blob support in
>> indexeddb?
>
> I can't speak for the indexedDB developers, but the last I heard
> on-list, they were at least hoping to support immutable Blobs, but not
> soon.
>
> More on those portability issues:
>
> We've been implementing the current FileSystem API spec in Chrome, and
> had planned that our implementation of the sandbox would be just a
> passthrough to a directory under the user's profile.  We've been
> testing that out, currently only allowing access to extensions and
> apps, but we've gotten feedback that long profile paths are causing a
> problem on Windows.
>
> Our current implementation on Windows uses the standard API calls that
> limit paths to 256 characters long.  If your profile directory is
> deeply nested [e.g. because you have a really long username], code
> that works for your friend might not work for you.  We thought we'd
> fix that by using the long-path APIs that allow creation of paths up
> to ~32k chars long.  However, it seems that, even though Windows has
> these APIs, most programs don't use them.  In particular, Windows
> Explorer doesn't, so it chokes on them and can't delete files in
> directories with long names.  We think it would be really awkward if
> Chrome were creating directories under its profile directory that
> couldn't easily be deleted, so we're going to start virtualizing our
> filesystem implementation.  We'll still have real native files for
> speed of access and ease of modification, but we'll be storing the
> paths in an internal database.
>
> This then brings up the question of whether one can specify a uniform,
> useful interface for the API that *doesn't* require virtualization of
> the filesystem, but I'll be bringing that up in another thread.
>
>     Eric
>

Re: [whatwg] Intent of the FileSystem API

[Jonas Sicking]

On Mon, Feb 28, 2011 at 3:50 PM, Charles Pritchard  wrote:
> Are there any plans of reviewing/visiting efficient File/Blob support in
> indexeddb?

Support for File and Blob is technically in the spec already, though I
don't think any implementations support it.

The more controversial part is supporting storing FileEntry or
something like it. This would allow storing references to files which
can be written to. I'm fairly sure we'll add experimental support for
that in Firefox soon (can't give any dates at this time). I haven't
really talked about it with other IndexedDB implementors, so I can't
give gauge how likely it is to appear in the spec.

/ Jonas

Re: [whatwg] Intent of the FileSystem API

[Charles Pritchard]

On 2/28/2011 3:50 PM, Ian Hickson wrote:

On Mon, 28 Feb 2011, Charles Pritchard wrote:

I'd like some clarification on the intent of the FileSystem API:
requestFileSystem permanent, getDirectory and getFile.

Are they intended to directly pass through to the host operating system, or
are they just programming constructs?

They're not particularly easy to use, compared to the IndexedDB data store,
but if they are OS level commands,
then they do help with usability of a web app with the rest of the OS: with
managing files directly from the OS file manager.

Is the FileSystem API intended to expose a folder on the host operating
system, or is it just an abstraction
which can be bolted onto any data system (such as, using SQLite as a backend,
as appCache and IndexedDB do)  ?

Are there any plans of reviewing/visiting efficient File/Blob support in
indexeddb?

You are likely to get a more useful response to your questions if you send
the above feedback to the lists mentioned in the specs as the place to
send feedback, namely public-webapps@w3.org:

| If you wish to make comments regarding this document, please send them
| to public-webapps@w3.org (subscribe, archives).
  -- http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
  -- http://dvcs.w3.org/hg/IndexedDB/raw-file/tip/Overview.html

HTH,

Re: FileSystem API: Avoiding Upload Forms And Temporary Downloads

[Eric Uhrhane]

On Mon, Nov 29, 2010 at 11:45 AM, Joran Greef  wrote:
> I have some questions regarding the FileSystem API:
>
> 1. It would be great to be able to let the user choose where they want their 
> sandboxed directory located for the web app, i.e. on the desktop for quick 
> access. That way they can drag files directly to the directory, which could 
> be used as a dropbox for synching to a server. Would this be possible (or at 
> least a mechanism to link to the directory wherever the browser may choose to 
> place it)? Otherwise, apps like Dropbox would not be possible in a browser.

Yes, we've certainly talked about this as a logical next step for the
API.  We want to make sure the basic security/usability/quota issues
with the current spec are dealt with before we expand to such use
cases, but I think there's a lot of great stuff that could be built on
this idea.  For example, you might want to give flickr.com access to
your photos directory.

> 2. It seems that dragging a file out of a web app is currently copy-on-write? 
> So you drag a file out the web app into Excel but subsequent changes in Excel 
> would be lost to the web app (it seems like it's already possible for the web 
> app to poll the sandboxed directory for file changes)? If so, it means that 
> the FileSystem API would force the following work flow: the user saves a temp 
> file somewhere (probably on the desktop) then re-uploads using a web form, 
> and then minimizes the browser and deletes the temp file, and then maximizes 
> the browser again? That would be a bad case of Fitts' Law and quickly become 
> a show-stopper if the user needs to frequently edit files using native 
> applications.

I don't see how Fitts' Law is relevant, but you raise an interesting
point about drag-and-drop.  When you drag a link out of an HTML
document, that's usually a drag of a link, not a file.  GMail
implements attachment drag-out, but that's a copy, not a path.  The
FileSystem API will give you URIs that you'd build a page around just
like any other; if you want different behavior than web URIs, we'd
have to change something.  I think you'd really want to implement this
by overriding drag+drop behavior rather than changing the file API,
but you may also need to give scripts access to the true underlying
paths in order to make that work, and there are security issues with
that.  I'll keep it in mind; if you have a specific proposal for an
API, please post it.

> 3. It must be possible to link to files within the sandboxed directory and 
> have them open in the default native application. I can understand that 
> .exe's need to be neutered, but content files such as .doc and .xls must have 
> a method for opening in the default application. Would this be possible? 
> Otherwise the only solution would be to trigger the Download window, creating 
> a temp file in the Downloads folder for a file that already exists on the 
> filesystem?

This runs into the same issues as #2--local URIs would get treated
differently than online URIs, we'd have to tell the applications the
true locations of the files, and we'd lose any protection provided by
the download manager.  I'm not going to say that this is impossible,
but it might require elevated privileges beyond those available to the
average web page, and not everyone would want it.  I'd like to see an
extension do this and see how it went before I'd expect the web
platform to allow it.

> 4. In my mind, the FileSystem Api has a shot at improving user experience by 
> helping to avoid file upload forms, and temp file downloads. I'm not sure 
> these goals are possible with the current spec?

I'm not sure what you're asking here.

> These use-cases may prove to be vital building blocks for the next wave of 
> networked applications and it would be great to see them in the new 
> FileSystem API.
>
> Regards
>
> Joran Greef
>
>

FileSystem API: Avoiding Upload Forms And Temporary Downloads

[Joran Greef]

I have some questions regarding the FileSystem API: 

1. It would be great to be able to let the user choose where they want their 
sandboxed directory located for the web app, i.e. on the desktop for quick 
access. That way they can drag files directly to the directory, which could be 
used as a dropbox for synching to a server. Would this be possible (or at least 
a mechanism to link to the directory wherever the browser may choose to place 
it)? Otherwise, apps like Dropbox would not be possible in a browser.

2. It seems that dragging a file out of a web app is currently copy-on-write? 
So you drag a file out the web app into Excel but subsequent changes in Excel 
would be lost to the web app (it seems like it's already possible for the web 
app to poll the sandboxed directory for file changes)? If so, it means that the 
FileSystem API would force the following work flow: the user saves a temp file 
somewhere (probably on the desktop) then re-uploads using a web form, and then 
minimizes the browser and deletes the temp file, and then maximizes the browser 
again? That would be a bad case of Fitts' Law and quickly become a show-stopper 
if the user needs to frequently edit files using native applications.

3. It must be possible to link to files within the sandboxed directory and have 
them open in the default native application. I can understand that .exe's need 
to be neutered, but content files such as .doc and .xls must have a method for 
opening in the default application. Would this be possible? Otherwise the only 
solution would be to trigger the Download window, creating a temp file in the 
Downloads folder for a file that already exists on the filesystem?

4. In my mind, the FileSystem Api has a shot at improving user experience by 
helping to avoid file upload forms, and temp file downloads. I'm not sure these 
goals are possible with the current spec?

These use-cases may prove to be vital building blocks for the next wave of 
networked applications and it would be great to see them in the new FileSystem 
API.

Regards

Joran Greef

Re: FileSystem API - overwrite flag for copy/move?

[Eric Uhrhane]

2010/9/16 Eric Uhrhane :
> How about this?
>
> For a move/copy of a file on top of existing file, or a directory on
> top of an existing empty directory, you get an automatic overwrite.
> A move/copy of a file on top of an existing directory, or of a
> directory on top of an existing file, will always fail.
> A move/copy of a file or directory on top of an existing non-empty
> directory will always fail.
>
> That matches Posix[1] rename behavior, and should cover most or all of
> the normal use cases.
> If necessary, we can consider adding a "don't overwrite" flag, but
> that may be difficult to implement without race conditions on all
> platforms.

I've added comments to this effect to the moveTo definitions.

> Regarding recursive deletion of directories:
>
> One option is to add a flag to remove(); that flag will be ignored if
> the Entry is a file, so it's not as clean as it might be, but it keeps
> the interface small.
> Another is to add a removeRecursively() method to DirectoryEntry; this
> makes it really clear what's going on, and might prevent some
> accidental deletions.

I've added removeRecursively to DirectoryEntry and DirectoryEntrySync.

> Which do you prefer?
>
> [1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html
>
> 2010/9/9 Kinuko Yasuda :
>> On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) 
>> wrote:
>>>
>>> I think recursive copy/remove is a very valid use case. As for overwrite,
>>> is a flag necessary? On most OSes you already get overwrite as the default
>>> behaviour (at least from APIs, many interactive UAs such as Explorer on
>>> windows will prompt), is there a compelling argument why it should be
>>> different for a web api?
>>
>> Making overwriting mode default for copy/move sounds reasonable to me too.
>>  Especially if we allow recursive remove (and I think there would be more
>> need for this one) it'd look more consistent.
>> As for providing options, I was a bit concerned about the complexity in
>> programming with nested callbacks, but in this case it's not a big deal
>> (we'll need only two) and wouldn't be a problem.
>> I'm more concerned with recursive remove though.  The js code to do that
>> isn't very pretty.
>>
>>> On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda 
>>> wrote:

 On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda 
 wrote:
>
> On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane  wrote:
>>
>> On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda 
>> wrote:
>> > Hi,
>> > I have a question about Entry.moveTo/copyTo behavior defined in
>> > the File API: Directories and System [1].
>> > Currently the API doesn't specify how Entry.moveTo() and copyTo()
>> > should
>> > behave
>> > when a source entry is a file and there's *already* a file at the
>> > destination path.
>> > Should UAs overwrite the existing file at the destination path or
>> > not?
>> > Or maybe we should add an 'overwrite' flag to indicate if the script
>> > wants
>> > to overwrite an existing file or not?
>>
>> I'm open to a flag.  We're already up to 4 parameters to each of those
>> methods, though...5 is a bit ungainly.  I'm concerned that we might
>> find another flag to add at some point, and we'd then be up to 6.
>> What about adding an flags object, as in getFile, to allow for
>> expansion?
>
> Adding a flag or flags object (suppose the other thread about Flags will
> be settled) sounds good to me.
> Or I think it's also ok to explicitly disallow overwriting copy/move,
> i.e. specify that 'it is an error to copy or move a file or directory if
> there's already an entry'.  In this case it might be better to have 
> another
> error code like ENTRY_EXISTS_ERR so that the user script can act
> differently.  (But in general having a handy option would make programming
> much easier in async context where every operation requires one or two
> callbacks.)
> If we're going to add 'overwrite' flag, there'll be a few more things to
> be made clear.
> For example I wonder how the overwriting copy/move should behave when
> there's already a **directory** at the destination path/name.
> Should the UA remove the existing directory and create a new entry at
> the same path?
> This sounds reasonable but it'll also provide a handy alternative way to
> remove a directory recursively.

 By the way how do you think about recursive remove?
 Is there a reason (or past discussion) not to have recursive option in
 remove?  (I mean, other than the fact that adding more and more options to
 every method doesn't look very clean.)
 I found that it's not very easy to remove a directory when there're
 children in it -- it requires multiple DirectoryReader.readEntries and
 Entry.remove in a nested way.
 Thanks,
>
> Or should the UA create a new entry *under* the directory?
> This

Re: FileSystem API - overwrite flag for copy/move?

[Kinuko Yasuda]

On Thu, Sep 16, 2010 at 4:00 PM, Eric Uhrhane  wrote:
> 2010/9/16 Kinuko Yasuda :
>> On Thu, Sep 16, 2010 at 2:50 PM, Eric Uhrhane  wrote:
>>> How about this?
>>>
>>> For a move/copy of a file on top of existing file, or a directory on
>>> top of an existing empty directory, you get an automatic overwrite.
>>> A move/copy of a file on top of an existing directory, or of a
>>> directory on top of an existing file, will always fail.
>>> A move/copy of a file or directory on top of an existing non-empty
>>> directory will always fail.
>>
>> This sounds good to me.
>> I think sticking to the compatibility with Posix would make sense to
>> developers too.
>>
>>> That matches Posix[1] rename behavior, and should cover most or all of
>>> the normal use cases.
>>> If necessary, we can consider adding a "don't overwrite" flag, but
>>> that may be difficult to implement without race conditions on all
>>> platforms.
>>
>> We can achieve this in a safer way by combining getFile/getDirectory
>> with exclusive flag and move/copy, so I think it's reasonable not to
>> have "don't overwrite" flag.
>>
>>> Regarding recursive deletion of directories:
>>>
>>> One option is to add a flag to remove(); that flag will be ignored if
>>> the Entry is a file, so it's not as clean as it might be, but it keeps
>>> the interface small.
>>> Another is to add a removeRecursively() method to DirectoryEntry; this
>>> makes it really clear what's going on, and might prevent some
>>> accidental deletions.
>>
>> I have no strong opinion on this but removeRecursively might be error prone?
>> (Recursive remove can be very destructive and it should be triggered
>> with an explicit intention.)
>
> Which do you think is more error-prone, a method called
> removeRecursively, or a boolean flag to remove?

Sorry I messed up the sentence...
I meant I prefer removeRecursively.

Thanks,

>>> Which do you prefer?
>>>
>>> [1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html
>>>
>>> 2010/9/9 Kinuko Yasuda :
 On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) 
 wrote:
>
> I think recursive copy/remove is a very valid use case. As for overwrite,
> is a flag necessary? On most OSes you already get overwrite as the default
> behaviour (at least from APIs, many interactive UAs such as Explorer on
> windows will prompt), is there a compelling argument why it should be
> different for a web api?

 Making overwriting mode default for copy/move sounds reasonable to me too.
  Especially if we allow recursive remove (and I think there would be more
 need for this one) it'd look more consistent.
 As for providing options, I was a bit concerned about the complexity in
 programming with nested callbacks, but in this case it's not a big deal
 (we'll need only two) and wouldn't be a problem.
 I'm more concerned with recursive remove though.  The js code to do that
 isn't very pretty.

> On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda 
> wrote:
>>
>> On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda 
>> wrote:
>>>
>>> On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane  wrote:

 On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda 
 wrote:
 > Hi,
 > I have a question about Entry.moveTo/copyTo behavior defined in
 > the File API: Directories and System [1].
 > Currently the API doesn't specify how Entry.moveTo() and copyTo()
 > should
 > behave
 > when a source entry is a file and there's *already* a file at the
 > destination path.
 > Should UAs overwrite the existing file at the destination path or
 > not?
 > Or maybe we should add an 'overwrite' flag to indicate if the script
 > wants
 > to overwrite an existing file or not?

 I'm open to a flag.  We're already up to 4 parameters to each of those
 methods, though...5 is a bit ungainly.  I'm concerned that we might
 find another flag to add at some point, and we'd then be up to 6.
 What about adding an flags object, as in getFile, to allow for
 expansion?
>>>
>>> Adding a flag or flags object (suppose the other thread about Flags will
>>> be settled) sounds good to me.
>>> Or I think it's also ok to explicitly disallow overwriting copy/move,
>>> i.e. specify that 'it is an error to copy or move a file or directory if
>>> there's already an entry'.  In this case it might be better to have 
>>> another
>>> error code like ENTRY_EXISTS_ERR so that the user script can act
>>> differently.  (But in general having a handy option would make 
>>> programming
>>> much easier in async context where every operation requires one or two
>>> callbacks.)
>>> If we're going to add 'overwrite' flag, there'll be a few more things to
>>> be made clear.
>>> For example I wonder how the overwriting copy/move

Re: FileSystem API - overwrite flag for copy/move?

[Eric Uhrhane]

2010/9/16 Kinuko Yasuda :
> On Thu, Sep 16, 2010 at 2:50 PM, Eric Uhrhane  wrote:
>> How about this?
>>
>> For a move/copy of a file on top of existing file, or a directory on
>> top of an existing empty directory, you get an automatic overwrite.
>> A move/copy of a file on top of an existing directory, or of a
>> directory on top of an existing file, will always fail.
>> A move/copy of a file or directory on top of an existing non-empty
>> directory will always fail.
>
> This sounds good to me.
> I think sticking to the compatibility with Posix would make sense to
> developers too.
>
>> That matches Posix[1] rename behavior, and should cover most or all of
>> the normal use cases.
>> If necessary, we can consider adding a "don't overwrite" flag, but
>> that may be difficult to implement without race conditions on all
>> platforms.
>
> We can achieve this in a safer way by combining getFile/getDirectory
> with exclusive flag and move/copy, so I think it's reasonable not to
> have "don't overwrite" flag.
>
>> Regarding recursive deletion of directories:
>>
>> One option is to add a flag to remove(); that flag will be ignored if
>> the Entry is a file, so it's not as clean as it might be, but it keeps
>> the interface small.
>> Another is to add a removeRecursively() method to DirectoryEntry; this
>> makes it really clear what's going on, and might prevent some
>> accidental deletions.
>
> I have no strong opinion on this but removeRecursively might be error prone?
> (Recursive remove can be very destructive and it should be triggered
> with an explicit intention.)

Which do you think is more error-prone, a method called
removeRecursively, or a boolean flag to remove?

>> Which do you prefer?
>>
>> [1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html
>>
>> 2010/9/9 Kinuko Yasuda :
>>> On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) 
>>> wrote:

 I think recursive copy/remove is a very valid use case. As for overwrite,
 is a flag necessary? On most OSes you already get overwrite as the default
 behaviour (at least from APIs, many interactive UAs such as Explorer on
 windows will prompt), is there a compelling argument why it should be
 different for a web api?
>>>
>>> Making overwriting mode default for copy/move sounds reasonable to me too.
>>>  Especially if we allow recursive remove (and I think there would be more
>>> need for this one) it'd look more consistent.
>>> As for providing options, I was a bit concerned about the complexity in
>>> programming with nested callbacks, but in this case it's not a big deal
>>> (we'll need only two) and wouldn't be a problem.
>>> I'm more concerned with recursive remove though.  The js code to do that
>>> isn't very pretty.
>>>
 On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda 
 wrote:
>
> On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda 
> wrote:
>>
>> On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane  wrote:
>>>
>>> On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda 
>>> wrote:
>>> > Hi,
>>> > I have a question about Entry.moveTo/copyTo behavior defined in
>>> > the File API: Directories and System [1].
>>> > Currently the API doesn't specify how Entry.moveTo() and copyTo()
>>> > should
>>> > behave
>>> > when a source entry is a file and there's *already* a file at the
>>> > destination path.
>>> > Should UAs overwrite the existing file at the destination path or
>>> > not?
>>> > Or maybe we should add an 'overwrite' flag to indicate if the script
>>> > wants
>>> > to overwrite an existing file or not?
>>>
>>> I'm open to a flag.  We're already up to 4 parameters to each of those
>>> methods, though...5 is a bit ungainly.  I'm concerned that we might
>>> find another flag to add at some point, and we'd then be up to 6.
>>> What about adding an flags object, as in getFile, to allow for
>>> expansion?
>>
>> Adding a flag or flags object (suppose the other thread about Flags will
>> be settled) sounds good to me.
>> Or I think it's also ok to explicitly disallow overwriting copy/move,
>> i.e. specify that 'it is an error to copy or move a file or directory if
>> there's already an entry'.  In this case it might be better to have 
>> another
>> error code like ENTRY_EXISTS_ERR so that the user script can act
>> differently.  (But in general having a handy option would make 
>> programming
>> much easier in async context where every operation requires one or two
>> callbacks.)
>> If we're going to add 'overwrite' flag, there'll be a few more things to
>> be made clear.
>> For example I wonder how the overwriting copy/move should behave when
>> there's already a **directory** at the destination path/name.
>> Should the UA remove the existing directory and create a new entry at
>> the same path?
>> This sounds reasonable but it'll also provide a h

Re: FileSystem API - overwrite flag for copy/move?

[Kinuko Yasuda]

On Thu, Sep 16, 2010 at 2:50 PM, Eric Uhrhane  wrote:
> How about this?
>
> For a move/copy of a file on top of existing file, or a directory on
> top of an existing empty directory, you get an automatic overwrite.
> A move/copy of a file on top of an existing directory, or of a
> directory on top of an existing file, will always fail.
> A move/copy of a file or directory on top of an existing non-empty
> directory will always fail.

This sounds good to me.
I think sticking to the compatibility with Posix would make sense to
developers too.

> That matches Posix[1] rename behavior, and should cover most or all of
> the normal use cases.
> If necessary, we can consider adding a "don't overwrite" flag, but
> that may be difficult to implement without race conditions on all
> platforms.

We can achieve this in a safer way by combining getFile/getDirectory
with exclusive flag and move/copy, so I think it's reasonable not to
have "don't overwrite" flag.

> Regarding recursive deletion of directories:
>
> One option is to add a flag to remove(); that flag will be ignored if
> the Entry is a file, so it's not as clean as it might be, but it keeps
> the interface small.
> Another is to add a removeRecursively() method to DirectoryEntry; this
> makes it really clear what's going on, and might prevent some
> accidental deletions.

I have no strong opinion on this but removeRecursively might be error prone?
(Recursive remove can be very destructive and it should be triggered
with an explicit intention.)

Thanks,

> Which do you prefer?
>
> [1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html
>
> 2010/9/9 Kinuko Yasuda :
>> On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) 
>> wrote:
>>>
>>> I think recursive copy/remove is a very valid use case. As for overwrite,
>>> is a flag necessary? On most OSes you already get overwrite as the default
>>> behaviour (at least from APIs, many interactive UAs such as Explorer on
>>> windows will prompt), is there a compelling argument why it should be
>>> different for a web api?
>>
>> Making overwriting mode default for copy/move sounds reasonable to me too.
>>  Especially if we allow recursive remove (and I think there would be more
>> need for this one) it'd look more consistent.
>> As for providing options, I was a bit concerned about the complexity in
>> programming with nested callbacks, but in this case it's not a big deal
>> (we'll need only two) and wouldn't be a problem.
>> I'm more concerned with recursive remove though.  The js code to do that
>> isn't very pretty.
>>
>>> On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda 
>>> wrote:

 On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda 
 wrote:
>
> On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane  wrote:
>>
>> On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda 
>> wrote:
>> > Hi,
>> > I have a question about Entry.moveTo/copyTo behavior defined in
>> > the File API: Directories and System [1].
>> > Currently the API doesn't specify how Entry.moveTo() and copyTo()
>> > should
>> > behave
>> > when a source entry is a file and there's *already* a file at the
>> > destination path.
>> > Should UAs overwrite the existing file at the destination path or
>> > not?
>> > Or maybe we should add an 'overwrite' flag to indicate if the script
>> > wants
>> > to overwrite an existing file or not?
>>
>> I'm open to a flag.  We're already up to 4 parameters to each of those
>> methods, though...5 is a bit ungainly.  I'm concerned that we might
>> find another flag to add at some point, and we'd then be up to 6.
>> What about adding an flags object, as in getFile, to allow for
>> expansion?
>
> Adding a flag or flags object (suppose the other thread about Flags will
> be settled) sounds good to me.
> Or I think it's also ok to explicitly disallow overwriting copy/move,
> i.e. specify that 'it is an error to copy or move a file or directory if
> there's already an entry'.  In this case it might be better to have 
> another
> error code like ENTRY_EXISTS_ERR so that the user script can act
> differently.  (But in general having a handy option would make programming
> much easier in async context where every operation requires one or two
> callbacks.)
> If we're going to add 'overwrite' flag, there'll be a few more things to
> be made clear.
> For example I wonder how the overwriting copy/move should behave when
> there's already a **directory** at the destination path/name.
> Should the UA remove the existing directory and create a new entry at
> the same path?
> This sounds reasonable but it'll also provide a handy alternative way to
> remove a directory recursively.

 By the way how do you think about recursive remove?
 Is there a reason (or past discussion) not to have recursive option in
 remove?  (I mean, other than the fact tha

Re: FileSystem API - overwrite flag for copy/move?

[Eric Uhrhane]

How about this?

For a move/copy of a file on top of existing file, or a directory on
top of an existing empty directory, you get an automatic overwrite.
A move/copy of a file on top of an existing directory, or of a
directory on top of an existing file, will always fail.
A move/copy of a file or directory on top of an existing non-empty
directory will always fail.

That matches Posix[1] rename behavior, and should cover most or all of
the normal use cases.
If necessary, we can consider adding a "don't overwrite" flag, but
that may be difficult to implement without race conditions on all
platforms.

Regarding recursive deletion of directories:

One option is to add a flag to remove(); that flag will be ignored if
the Entry is a file, so it's not as clean as it might be, but it keeps
the interface small.
Another is to add a removeRecursively() method to DirectoryEntry; this
makes it really clear what's going on, and might prevent some
accidental deletions.

Which do you prefer?

[1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html

2010/9/9 Kinuko Yasuda :
> On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) 
> wrote:
>>
>> I think recursive copy/remove is a very valid use case. As for overwrite,
>> is a flag necessary? On most OSes you already get overwrite as the default
>> behaviour (at least from APIs, many interactive UAs such as Explorer on
>> windows will prompt), is there a compelling argument why it should be
>> different for a web api?
>
> Making overwriting mode default for copy/move sounds reasonable to me too.
>  Especially if we allow recursive remove (and I think there would be more
> need for this one) it'd look more consistent.
> As for providing options, I was a bit concerned about the complexity in
> programming with nested callbacks, but in this case it's not a big deal
> (we'll need only two) and wouldn't be a problem.
> I'm more concerned with recursive remove though.  The js code to do that
> isn't very pretty.
>
>> On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda 
>> wrote:
>>>
>>> On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda 
>>> wrote:

 On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane  wrote:
>
> On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda 
> wrote:
> > Hi,
> > I have a question about Entry.moveTo/copyTo behavior defined in
> > the File API: Directories and System [1].
> > Currently the API doesn't specify how Entry.moveTo() and copyTo()
> > should
> > behave
> > when a source entry is a file and there's *already* a file at the
> > destination path.
> > Should UAs overwrite the existing file at the destination path or
> > not?
> > Or maybe we should add an 'overwrite' flag to indicate if the script
> > wants
> > to overwrite an existing file or not?
>
> I'm open to a flag.  We're already up to 4 parameters to each of those
> methods, though...5 is a bit ungainly.  I'm concerned that we might
> find another flag to add at some point, and we'd then be up to 6.
> What about adding an flags object, as in getFile, to allow for
> expansion?

 Adding a flag or flags object (suppose the other thread about Flags will
 be settled) sounds good to me.
 Or I think it's also ok to explicitly disallow overwriting copy/move,
 i.e. specify that 'it is an error to copy or move a file or directory if
 there's already an entry'.  In this case it might be better to have another
 error code like ENTRY_EXISTS_ERR so that the user script can act
 differently.  (But in general having a handy option would make programming
 much easier in async context where every operation requires one or two
 callbacks.)
 If we're going to add 'overwrite' flag, there'll be a few more things to
 be made clear.
 For example I wonder how the overwriting copy/move should behave when
 there's already a **directory** at the destination path/name.
 Should the UA remove the existing directory and create a new entry at
 the same path?
 This sounds reasonable but it'll also provide a handy alternative way to
 remove a directory recursively.
>>>
>>> By the way how do you think about recursive remove?
>>> Is there a reason (or past discussion) not to have recursive option in
>>> remove?  (I mean, other than the fact that adding more and more options to
>>> every method doesn't look very clean.)
>>> I found that it's not very easy to remove a directory when there're
>>> children in it -- it requires multiple DirectoryReader.readEntries and
>>> Entry.remove in a nested way.
>>> Thanks,

 Or should the UA create a new entry *under* the directory?
 This behavior doesn't sound like 'overwriting'.  The resulting path will
 be like 'destParentPath/name/name' which doesn't sound quite consistent 
 with
 the spec either.

> > Similarly I wondered if we'd want to have a 'recursive' flag for
> > moveTo/copyTo.
> > I thi

Re: Flags for getFile/getDirectory (Re: New draft of FileSystem API posted)

[Kinuko Yasuda]

Sounds good, going back to the original topic, I'm convinced that the
current specification (Flags object) is more JavaScript-ish and now I see no
reason to change it.

Thanks,
Kinuko

On Wed, Sep 8, 2010 at 11:03 AM, Eric Uhrhane  wrote:

> On Tue, Sep 7, 2010 at 11:13 PM, Ian Hickson  wrote:
> > On Tue, 7 Sep 2010, Darin Fisher wrote:
> >>
> >> Based on the ValidityState example, it seems that the members of Flags
> >> should be camelCase then instead of UPPERCASE?
> >
> > The platform convention, insofar as there is a convention, is that
> > constants are uppercase, members are camelCase, and interfaces and
> > constructors are TitleCase.
>
> I was thinking of those as constants, but really they're members, so
> yes, they should be camelCase.  I'll fix them.
>
> > --
> > Ian Hickson   U+1047E)\._.,--,'``.fL
> > http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
> > Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
> >
>

Re: FileSystem API - overwrite flag for copy/move?

[Kinuko Yasuda]

On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) wrote:

> I think recursive copy/remove is a very valid use case. As for overwrite,
> is a flag necessary? On most OSes you already get overwrite as the default
> behaviour (at least from APIs, many interactive UAs such as Explorer on
> windows will prompt), is there a compelling argument why it should be
> different for a web api?


Making overwriting mode default for copy/move sounds reasonable to me too.
 Especially if we allow recursive remove (and I think there would be more
need for this one) it'd look more consistent.

As for providing options, I was a bit concerned about the complexity in
programming with nested callbacks, but in this case it's not a big deal
(we'll need only two) and wouldn't be a problem.

I'm more concerned with recursive remove though.  The js code to do that
isn't very pretty.


On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda  wrote:
>
>> On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda wrote:
>>
>>> On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane  wrote:
>>>
 On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda 
 wrote:
 > Hi,
 > I have a question about Entry.moveTo/copyTo behavior defined in
 > the File API: Directories and System [1].
 > Currently the API doesn't specify how Entry.moveTo() and copyTo()
 should
 > behave
 > when a source entry is a file and there's *already* a file at the
 > destination path.
 > Should UAs overwrite the existing file at the destination path or not?
 > Or maybe we should add an 'overwrite' flag to indicate if the script
 wants
 > to overwrite an existing file or not?

 I'm open to a flag.  We're already up to 4 parameters to each of those
 methods, though...5 is a bit ungainly.  I'm concerned that we might
 find another flag to add at some point, and we'd then be up to 6.
 What about adding an flags object, as in getFile, to allow for
 expansion?
>>>
>>>
>>> Adding a flag or flags object (suppose the other thread about Flags will
>>> be settled) sounds good to me.
>>>
>>> Or I think it's also ok to explicitly disallow overwriting copy/move,
>>> i.e. specify that 'it is an error to copy or move a file or directory if
>>> there's already an entry'.  In this case it might be better to have another
>>> error code like ENTRY_EXISTS_ERR so that the user script can act
>>> differently.  (But in general having a handy option would make programming
>>> much easier in async context where every operation requires one or two
>>> callbacks.)
>>>
>>> If we're going to add 'overwrite' flag, there'll be a few more things to
>>> be made clear.
>>> For example I wonder how the overwriting copy/move should behave when
>>> there's already a **directory** at the destination path/name.
>>>
>>> Should the UA remove the existing directory and create a new entry at the
>>> same path?
>>> This sounds reasonable but it'll also provide a handy alternative way to
>>> remove a directory recursively.
>>>
>>
>> By the way how do you think about recursive remove?
>> Is there a reason (or past discussion) not to have recursive option in
>> remove?  (I mean, other than the fact that adding more and more options to
>> every method doesn't look very clean.)
>>
>> I found that it's not very easy to remove a directory when there're
>> children in it -- it requires multiple DirectoryReader.readEntries and
>> Entry.remove in a nested way.
>>
>> Thanks,
>>
>> Or should the UA create a new entry *under* the directory?
>>> This behavior doesn't sound like 'overwriting'.  The resulting path will
>>> be like 'destParentPath/name/name' which doesn't sound quite consistent with
>>> the spec either.
>>>
>>>
>>> > Similarly I wondered if we'd want to have a 'recursive' flag for
 > moveTo/copyTo.
 > I think for directories we can simply assume that the user wants to
 > move/copy
 > them recursively, but it might be good to add some notion about that
 in the
 > spec.

 How about I add a note indicating that directory copies are always
 recursive?
 I don't think we need anything for move.
>>>
>>>
>>> This sounds good to me.  Thanks!
>>>
>>>
>>>
  > Thanks,
 > [1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
 >

>>>
>>>
>>
>

Re: FileSystem API - overwrite flag for copy/move?

[イアンフェッティ]

I think recursive copy/remove is a very valid use case. As for overwrite, is
a flag necessary? On most OSes you already get overwrite as the default
behaviour (at least from APIs, many interactive UAs such as Explorer on
windows will prompt), is there a compelling argument why it should be
different for a web api?

On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda  wrote:

> On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda wrote:
>
>> On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane  wrote:
>>
>>> On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda 
>>> wrote:
>>> > Hi,
>>> > I have a question about Entry.moveTo/copyTo behavior defined in
>>> > the File API: Directories and System [1].
>>> > Currently the API doesn't specify how Entry.moveTo() and copyTo()
>>> should
>>> > behave
>>> > when a source entry is a file and there's *already* a file at the
>>> > destination path.
>>> > Should UAs overwrite the existing file at the destination path or not?
>>> > Or maybe we should add an 'overwrite' flag to indicate if the script
>>> wants
>>> > to overwrite an existing file or not?
>>>
>>> I'm open to a flag.  We're already up to 4 parameters to each of those
>>> methods, though...5 is a bit ungainly.  I'm concerned that we might
>>> find another flag to add at some point, and we'd then be up to 6.
>>> What about adding an flags object, as in getFile, to allow for
>>> expansion?
>>
>>
>> Adding a flag or flags object (suppose the other thread about Flags will
>> be settled) sounds good to me.
>>
>> Or I think it's also ok to explicitly disallow overwriting copy/move, i.e.
>> specify that 'it is an error to copy or move a file or directory if there's
>> already an entry'.  In this case it might be better to have another error
>> code like ENTRY_EXISTS_ERR so that the user script can act differently.
>>  (But in general having a handy option would make programming much easier in
>> async context where every operation requires one or two callbacks.)
>>
>> If we're going to add 'overwrite' flag, there'll be a few more things to
>> be made clear.
>> For example I wonder how the overwriting copy/move should behave when
>> there's already a **directory** at the destination path/name.
>>
>> Should the UA remove the existing directory and create a new entry at the
>> same path?
>> This sounds reasonable but it'll also provide a handy alternative way to
>> remove a directory recursively.
>>
>
> By the way how do you think about recursive remove?
> Is there a reason (or past discussion) not to have recursive option in
> remove?  (I mean, other than the fact that adding more and more options to
> every method doesn't look very clean.)
>
> I found that it's not very easy to remove a directory when there're
> children in it -- it requires multiple DirectoryReader.readEntries and
> Entry.remove in a nested way.
>
> Thanks,
>
> Or should the UA create a new entry *under* the directory?
>> This behavior doesn't sound like 'overwriting'.  The resulting path will
>> be like 'destParentPath/name/name' which doesn't sound quite consistent with
>> the spec either.
>>
>>
>> > Similarly I wondered if we'd want to have a 'recursive' flag for
>>> > moveTo/copyTo.
>>> > I think for directories we can simply assume that the user wants to
>>> > move/copy
>>> > them recursively, but it might be good to add some notion about that in
>>> the
>>> > spec.
>>>
>>> How about I add a note indicating that directory copies are always
>>> recursive?
>>> I don't think we need anything for move.
>>
>>
>> This sounds good to me.  Thanks!
>>
>>
>>
>>>  > Thanks,
>>> > [1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
>>> >
>>>
>>
>>
>