Re: [fileapi] Pull Request on GitHub
I won't be editing it either. On Tue, Aug 16, 2016 at 4:44 AM, Marcos Cacereswrote: > On August 16, 2016 at 6:31:31 PM, Zhen Zhang (izgz...@gmail.com) wrote: > > Hi, > > > > I have a PR on GitHub regarding some issues of wording in current File > API spec: https://github.com/w3c/FileAPI/pull/42 > > , but nobody ever responded me there. > > I wonder if I should discuss the patch somewhere else? > > It seems that no one has touched that API for about 8 months. > > Marijn, are you still editing that document? I guess Jonas won't be, > but not sure about Arun. > >
Re: Directory Upload Proposal
On Fri, Apr 24, 2015 at 2:28 AM, Ali Alabbas a...@microsoft.com wrote: If there is sufficient interest, I would like to work on this within the scope of the WebApps working group. And I'll help with the FileSystem API bit, ensuring that full spec.[3] has bits about the Directory Upload proposal (outside of the sandbox). There's an HTML WG bit, too, as Jonas and Anne v K point out. I'm not sure what the best way to tackle that is, but I think a bug on HTML plus nudging over IRC over attribute names will go a fair distance. -- A*
flush() | was Re: FileSystem API Comments
Greetings Ali! I’ve been thinking about the discussion of flush(), and would like to see if I can make my previous statement a bit more nuanced. It turns out that flush() (in the vein of fsync/sync) is pretty useful, and after discussion with a few folks within Mozilla, I realize that it isn’t as simple as tacking it on to the “write-family” of Promises — as you point out, it is a potentially expensive operation. Something like a flush feature might help the following use cases: 1. Creating a database technology on top of the filesystem technology. This might include IndexedDB, but also WebSQL (as a hypothetical example). Most transactional operations like this need the ability to do something like flush. 2. Then, there’s the use case of compiling C++ codebases to JS. Well-known examples of this are games, leveraging asm.js. In this genre of use case, sometimes a large database is brought over (e.g. sqlite). It could be memory backed, but it is a definite bonus if it could be filesystem backed. Something like flush helps make that a possibility. Now the question is how to do this in a WebAPI, allowing for the power along with the mitigations that a web app might need, notably for performance? A few ideas below: On Oct 21, 2014, at 4:36 PM, Ali Alabbas a...@microsoft.com wrote: * flush() - This is costly functionality to expose and is likely to be overused by callers. It would be beneficial to automatically flush changes to disk by allowing the default file write behavior by the OS. For example, on Windows, we would leave it up to the filesystem cache to determine the best time to flush to disk. This is non-deterministic from the app's point of view, but the only time it is a potential problem is when there's a hard power-off. Most apps should not be concerned with this; only apps that have very high data reliability requirements would need the granular control of flushing to disk. In those cases a developer should use IndexedDB. So we should consider obscuring this functionality since it's not a common requirement and has a performance impact if it's widely used. I agree with the idea of obscuring the functionality a bit, especially given that it might not be necessary for a large class of operations. A few ways to do that: 1. Add this to a dictionary option when coining the FileHandleWritable from the Directory (e.g. add it to something like the OpeWriteOptions: http://w3c.github.io/filesystem-api/Overview.html#widl-Directory-openWrite-Promise-FileHandleWritable--DOMString-File-path-OpenWriteOptions-options). This way, the developer has the ability to “coin” a “more expensive” promise, if that particular set of write operations needs this feature. 2. Add this to the set of options on the FileHandleWritable. This could be by dictionary, again. Or, it could be a boolean on the FileHandleWritable’s write(). This latter might not be specific enough. Like other implementations, ours is not going to buffer anything, but rely on the underlying operating system’s buffer for writes and reads. 3. Stick with the idea of a method, like flush(). In this case, we might have to caveat the use of this, since the possibility of inexperienced developer misuse is high :-) It might help to see if we can determine some boundaries on this. Any feedback on some of these options would be valuable. I am thinking of 1. and 2. — A*
FileAPI | Programmatic Triggering of Downloads
WebAppsWG, At the recent F2F, I received an ACTION to strike one of the use cases from the File API, since the File API itself did not deliver on that use case. This is ACTION 750: https://www.w3.org/2008/webapps/track/actions/750 which asks me to remove the use case, since there isn’t a mechanism to trigger programmatic downloads to the underlying OS filesystem. But upon reflection, although FileAPI itself hasn’t provided any API surface for this, the recently finished HTML specification has given us something useful, but not implemented widely yet: http://www.w3.org/TR/html5/links.html#attr-hyperlink-download The use case itself calls for things such as storing assets for later use, including for use with third-party apps such as Calendar apps, which I’ve demonstrated using the Blob constructor and the “download” feature, since not everyone present at the meeting knew you could do this: http://aruner.net/resources/BlobURL/datePickerForCal.html In Firefox and Chrome (desktop) it works by triggering a download and invoking the Calendar application on your particular OS (Outlook 2013 on Windows or Calendar on Mac OS X). Safari shows the iCalendar entry “in the raw” and although Safari on iOS allows images that are Blob URLs to be downloaded, it doesn’t allow much to be done with other data types that are blob: URLs, such as iCalendar entries. I don’t think I should remove the use case, but instead said that the actual requirement was handled in HTML, even if implementation status isn’t at 100%. The File API provides additional guidance for File objects: http://dev.w3.org/2006/webapi/FileAPI/#processing-media-types — A*
Re: [FileAPI] Seeking status and plans [Was: [TPAC2014] Creating focused agenda]
On Oct 22, 2014, at 8:05 AM, Arthur Barstow art.bars...@gmail.com wrote: * File API: Arun and Jonas; which v1 bugs are blocking a new LC; what are next steps; timeline for LC. Arun, Jonas, Please see the above and respond accordingly. I am especially interested in the File API status but please also include a summary of your plans for the the Filesystem API. 1. File API: happy to move towards LC. Only active bug is a platform bug: https://www.w3.org/Bugs/Public/show_bug.cgi?id=26603 which applies to FileAPI’s use of the incumbent settings object for origin determination of Blob URLs. 2. FileSystem API: http://w3c.github.io/filesystem-api/Overview.html Plan: get feedback from implementors, fix the spec and resolve the spec.’s issues. Thanks to Ali Alabbas from the IE Platform Team for some early feedback. — A*
Re: FileSystem API Comments
Ali, First, thanks for your timely comments :) I’m in the process of editing the FileSystem API. Responses inline: On Oct 21, 2014, at 4:36 PM, Ali Alabbas a...@microsoft.com wrote: 1.1 Use cases (3. Audio/Photo editor with offline access or local cache for speed) * Edited files should be accessible by other client-side applications - Having the sandboxed file system share its contents between all apps would allow apps to tamper with the files of another app. snip / Admitedly, these use cases have been borrowed from the “File API: Directories and System” specification (which is now a W3C Note), at least for the purpose of providing equivalent functionality. In practice, everything you’ve pointed out makes it a hard problem to solve. The per-origin sandbox model also raises file lock issues on multiple access, but they are probably easier to solve, and not as prevalent. Also, we’re going to forego the “temporary” and “persistent” distinctions that are in the draft I think. And while there’s a technical dependency on Quota Manager, I don’t think there’s a spec. dependency in terms of API. Of course, certain Directory operations may reject a promise with a quota error. 3. The Directory Interface * Change events - I would like to revisit the discussion on apps getting notifications of changes to files/directories. This is a good point; right now there’s no way to do this. I’m open to suggestions. An early scratch pad version of my spec. changes proposed Directory as an EventTarget also, but… this won’t work for a variety of reasons. * removeDeep() move() - Do these support links or junctions? If not, what is the expected behavior? No; but the entire API doesn’t support these right now. * enumerate() - It would be useful to have pre-filtering support for the following: file/directory, ranges, wildcard search. Currently we would be forced to enumerate the entire set and manually filter out the items we want from the result set. I completely agree this would be useful, but there’s a problem to solve even before we get there! Right now, we say that we’ll fullfill the enumerate promise with something called “EventStream” which was initially a Tab Atkins proposal, and which would be really useful to get right. It’s underspecified right now, but I’m a fan of it ;-) We’ll have to think about how to return wildcard searches, etc., and how to annotate results in the result set. 4. The FileHandle Interface * FileHandles - Is this basically going to be the first to get the handle gets to use it and all subsequent calls need to wait for the file handle to become available again? Are there more details about the locking model used here? Yes; essentially the “first invocation” uses then releases it. A version of this problem was encountered when specifying FileReader (http://dev.w3.org/2006/webapi/FileAPI/#dfn-filereader) which used the internal state (but also accessible to the developer) “LOADING” to prevent multiple concurrent reads. * Auto-closing of FileHandles - This may cause confusion as it does not match the common developer mental model of a file handle which is “opened” and then available for use until it's “closed”. Perhaps it would be advantageous to have an explicit close function as part of the FileHandle interface? There are pros and cons either way. I’d be interested in solving this for the lion’s share of use cases. I’m not strongly opinionated on the matter of an explicit close function (we have one on Blob, for example), but it seems even this has drawbacks. * AbortableProgressPromise - It is not clear how a developer would define the abort callback of an AbortableProgressPromise. It seems that the user agent would be responsible for setting the abort callback since it instantiates and returns the AbortableProgressPromise. We’re going to not use an AbortableProgressPromise, but we will probably have a new beast called CancelablePromise. 5. The FileHandleWritable Interface * write() flush() - It might be useful to have support for “transacted” streams where the caller can write to a copy of the file and then have it atomically replaced: swap the old file with the new one and then delete the old file. Agreed. * flush() - This is costly functionality to expose and is likely to be overused by callers. Agreed — let’s flush flush(). 6. FileSystem Configuration Parameters * Dictionary DestinationDict - The DestinationDict seems to exist to facilitate the renaming of a directory in conjunction with a move. However, the same operation is done differently for files which makes the functionality non-uniform. Perhaps we can add a rename() function to make it more intuitive? I’ll commit to sample code and more “spec text” to make this clearer in my next
Re: File API: reading a Blob
On Aug 11, 2014, at 7:24 AM, Anne van Kesteren ann...@annevk.nl wrote: Other than “chunks of bytes” which needs some normative backbone, is the basic abstract model what you had in mind? If so, that might be worth getting into File APi and calling it done, because that’s a reusable abstract model for Fetch and for FileSystem. Yeah that looks good. https://whatwg.github.io/streams/ defines chunks and such, but is not quite there yet. But it is what we want to build this upon eventually. I’ve fixed the model according to these discussions: 1. We’ve separated the read operation and the “annotated task read operation.” The former uses Fetch’s notion of a body, which is a byte stream, and so is convenient to work with. Hopefully, it should allow plugging right back into Fetch. 2. The methods now invoke an annotated task read operation. http://dev.w3.org/2006/webapi/FileAPI/#reading-data-section — A*
Re: File API: reading a Blob
On Sep 3, 2014, at 6:02 PM, Aymeric Vitte vitteayme...@gmail.com wrote: The fact is that most of the W3C groups impacted by streams (File, indexedDB, MSE, WebRTC, WebCrypto, Workers, XHR, WebSockets, Media Stream, etc, I must forget a lot here) seem not to care a lot about it and maybe just expect streams to land in the right place in the APIs when they are available, by some unknown magic. I care about it. Till the API is totally baked, I’m amenable to getting the model right. File API now refers to chunks read, which is more correct. But I understand that your use cases aren’t catered to just yet; FileReader/FileReaderSync don’t do easily extractable partials. I’d like to see if there’s interest in the earlier proposal, to extract a stream straight from Blob. I still think that the effort should start from now for all the APIs (as well as the implementation inside browsers, which apparently has started for Chrome, but Chrome was supposed to have started some implementation of the previous Streams APIs, so it's not very clear), and that it should be very clearly synchronized, disregarding vague assumptions from the groups about low/high level and Vx releases, eluding the issue. What issue is being eluded? Seems like another of your main use cases is to have URL.createObjectURL or URL.createFor return a streamable resource. I agree that’s a good problem to solve. — A*
Re: File API: reading a Blob
Welcome back - we missed you :-) On Aug 5, 2014, at 9:43 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, Jul 17, 2014 at 2:58 PM, Arun Ranganathan a...@mozilla.com wrote: There are two questions: 1. How should FileReaderSync behave, to solve the majority of use cases? 2. What is a useful underlying abstraction for spec. authors that can be reused in present APIs like Fetch and future APIs? I'm not sure. I strongly think we should leave FileReaderSync and FileReader alone. Also note that FileReaderSync and XHR (sync) are not different, in that both don’t do partial data. But we should have a stream api that evolves to read, and it might be something off Blob itself. That leaves us the next problem, and what I think is the last problem in File API: Yeah, I now think that we want something even lower-level and build the task queuing primitive on top of that. (Basically by observing the stream that is being read and queuing tasks as data comes in, similar to Fetch. The synchronous case would just wait for the stream to complete. If I understand you correctly, you mean something that might be two-part (some hand waving below, but …): To read a Blob object /blob/, run these steps: 1. Let /s/ be a new buffer. 2. Return /s/, but continue running these steps asynchronously. 3. While /blob/'s underlying data stream is not closed, run these substeps: 1. Let /bytes/ be the result of reading a chunk from /blob/'s underlying data. 2. If /bytes/ is not failure, push /bytes/ to /s/ and set /s/'s transmitted to /bytes/'s length. 3. Otherwise, signal some kind of error to /s/ and terminate these steps. AND To read a Blob object with tasks: 1. Run the read a Blob algorithm above. 2. When reading the first /bytes/ queue a task called process read. 3. When pushing /bytes/ to /s/, queue a task called process read data. 4. When all /bytes/ are pushed to /s/ queue a task called process read EOF. 5. If an error condition is signaled queue a task called process error with a failure reason. Is “chunk” implementation defined? Right now we assume 1 byte or 50ms. “Chunk” seems a bit hand-wavy and hard to enforce, but… it might be the right approach. Would have to discuss with Domenic, but something like chunks seems to be much closer to how these things actually work. Other than “chunks of bytes” which needs some normative backbone, is the basic abstract model what you had in mind? If so, that might be worth getting into File APi and calling it done, because that’s a reusable abstract model for Fetch and for FileSystem. — A*
Re: File API: reading a Blob
Aymeric, (Sorry for the delay; I was traveling for the last part of July.) On Jul 17, 2014, at 11:25 AM, Aymeric Vitte vitteayme...@gmail.com wrote: I don't get this, most reads inside browsers are about fetching, and fetching does increment a resource while it is loaded/displayed (image, video, crypto operations, etc) And it's question of the fetch API in this thread. XHR has the same issue, fortunately you can use XHR with Range, now I think this issue should stop propagating What I meant is that while I don’t think FileReader *or* FileReaderSync should be modified *at this stage* to include partial data, there is a workaround: 1. After obtaining the Blob or File, use .slice() to generate something like a mySlices[ ] array of e.g. 2K each slice. 2. Then use fileReader.readAsArrayBuffer(mySlices[0]…). This should be relatively efficient, and in fact can be cued to work at the completion of the last read, based on the final event being fired (or something similar — you get the idea). This might be similar to XHR range. This at least allows chunked reads. I agree that we need a stream-based API, but I don’t agree that it should be FileReader or FileReaderSync. That’s simply doing too much with too little. However, I think the next step would be to generate stream-based reads directly off the Blob, e.g. have something like myBlob.read(…) which returns a stream, which can be cued with API calls. 1. Decoding was an issue with *readAsText*. I suppose we could make that method alone be all or nothing. I don't see where the issue can be, it should behave like textdecoder and reinject the undecoded bytes to the next chunk, personnaly I find really useless to offer the 'text' methods in files/fetching APIs. The issue is that it isn’t possible to know the total number of decoded bytes is till the whole file is decoded, but of course we could only represent the bytes read thus far in progress events. Again, I don’t think FileReader or FileReaderSync are the right APIs for that. — A*
Re: File API: reading a Blob
Aymeric, (Sorry for the delay; I was traveling for the last part of July.) On Jul 17, 2014, at 11:25 AM, Aymeric Vitte vitteayme...@gmail.com wrote: I don't get this, most reads inside browsers are about fetching, and fetching does increment a resource while it is loaded/displayed (image, video, crypto operations, etc) And it's question of the fetch API in this thread. XHR has the same issue, fortunately you can use XHR with Range, now I think this issue should stop propagating What I meant is that while I don’t think FileReader *or* FileReaderSync should be modified *at this stage* to include partial data, there is a workaround: 1. After obtaining the Blob or File, use .slice() to generate something like a mySlices[ ] array of e.g. 2K each slice. 2. Then use fileReader.readAsArrayBuffer(mySlices[0]…). This should be relatively efficient, and in fact can be cued to work at the completion of the last read, based on the final event being fired (or something similar — you get the idea). This might be similar to XHR range. This at least allows chunked reads. I agree that we need a stream-based API, but I don’t agree that it should be FileReader or FileReaderSync. That’s simply doing too much with too little. However, I think the next step would be to generate stream-based reads directly off the Blob, e.g. have something like myBlob.read(…) which returns a stream, which can be cued with API calls. 1. Decoding was an issue with *readAsText*. I suppose we could make that method alone be all or nothing. I don't see where the issue can be, it should behave like textdecoder and reinject the undecoded bytes to the next chunk, personnaly I find really useless to offer the 'text' methods in files/fetching APIs. The issue is that it isn’t possible to know the total number of decoded bytes is till the whole file is decoded, but of course we could only represent the bytes read thus far in progress events. Again, I don’t think FileReader or FileReaderSync are the right APIs for that. — A*
Re: File API: reading a Blob
On Jul 14, 2014, at 3:47 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, Jul 10, 2014 at 7:05 PM, Arun Ranganathan a...@mozilla.com wrote: On Jul 3, 2014, at 10:50 AM, Anne van Kesteren ann...@annevk.nl wrote: That would mean you would get different results between using FileReaderSync and XMLHttpRequest. That does not seem ideal The implementation train has already left the station on this. The movitation of an “ideal match-up with XMLHttpRequest doesn’t seem strong enough to revisit this by filing browser bugs across implementations (but Cc’ing K. Huey also). Well, surely if we support both, we'd like them to work in the same way so they can share the same underlying abstraction. There are two questions: 1. How should FileReaderSync behave, to solve the majority of use cases? 2. What is a useful underlying abstraction for spec. authors that can be reused in present APIs like Fetch and future APIs? I don’t think it is necessary to mix the two questions for APIs that are already shipping. Do you think that FileReaderSync AND FileReader should support partial Blob data in read results? Or that stream-based reads should do this with a different API? We agreed some time ago to not have partial data. Pointer? I also don't really see how that makes sense given how asynchronous read would perform. Well, the bug that removed them is: https://www.w3.org/Bugs/Public/show_bug.cgi?id=23158 and dates to last year. Problems really include decoding strings according to the encoding determination for incomplete Blobs: http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0063.html Another thread covered deltas in progress events: http://lists.w3.org/Archives/Public/public-webapps/2013JanMar/0069.html I don’t have pointers to IRC conversations, but: 1. Decoding was an issue with *readAsText*. I suppose we could make that method alone be all or nothing. 2. Use cases. MOST reads are useful with all data, after which you could use Blob manipulation (e.g. *slice*). New result objects each time didn’t seem optimal. And, it was something punted to Streams since that seemed like a longer term direction. There was also the idea of a Promise-based File API that could be consumed by the FileSystem API. But it might be useful to have an abstract read that helps Fetch and other things like Streams also: Yeah, I now think that we want something even lower-level and build the task queuing primitive on top of that. (Basically by observing the stream that is being read and queuing tasks as data comes in, similar to Fetch. The synchronous case would just wait for the stream to complete. If I understand you correctly, you mean something that might be two-part (some hand waving below, but …): To read a Blob object /blob/, run these steps: 1. Let /s/ be a new buffer. 2. Return /s/, but continue running these steps asynchronously. 3. While /blob/'s underlying data stream is not closed, run these substeps: 1. Let /bytes/ be the result of reading a chunk from /blob/'s underlying data. 2. If /bytes/ is not failure, push /bytes/ to /s/ and set /s/'s transmitted to /bytes/'s length. 3. Otherwise, signal some kind of error to /s/ and terminate these steps. AND To read a Blob object with tasks: 1. Run the read a Blob algorithm above. 2. When reading the first /bytes/ queue a task called process read. 3. When pushing /bytes/ to /s/, queue a task called process read data. 4. When all /bytes/ are pushed to /s/ queue a task called process read EOF. 5. If an error condition is signaled queue a task called process error with a failure reason. Is “chunk” implementation defined? Right now we assume 1 byte or 50ms. “Chunk” seems a bit hand-wavy and hard to enforce, but… it might be the right approach. — A*
Re: File API: reading a Blob
Aymeric, On Jul 16, 2014, at 8:20 AM, Aymeric Vitte vitteayme...@gmail.com wrote: Example: var myfile=new Blob(); //chunks are coming in myfile=new Blob([myfile,chunk],...) //mylink A tag mylink.href=URL.createObjectURL(myfile) click on mylink -- does not work Expected behavior: the file (a video for example) should play as it is incremented. This is inconsistent with the standard files behavior (see [1] for example), this example should work without having to use the Media Source Extensions API. This is a great use case, but breaks File API currently (and the abstract model, too). Options to consider might be: 1. Including partial Blob data with the existing FIle API. But we already ship as is, so we could forego decoding to get this right, but that doesn’t seem optimal either. 2. Use a new API for this, such as Streams. Since we’ll use streams for non-file cases, it could be useful here (and might be a better fit for the use case, which features heavy video use). — A*
Re: File API: Blob URL origin
On Jun 30, 2014, at 7:13 PM, Glenn Maynard gl...@zewt.org wrote: Why would the identifier not just be the blob URL itself? Done. Also, both Chrome and Firefox treat the entire URL as case-sensitive, eg. Blob:... won't revoke the URL, or uppercasing the hostname portion in Chrome. Using the whole URL as the identifier makes this easy to do. Subsequent attempts to dereference url must return a network error should be removed. That should already be the consequence of unregistering the URL, so this is a redundant requiremen Done: http://dev.w3.org/2006/webapi/FileAPI/ — A*
Re: File API: reading a Blob
On Jul 3, 2014, at 10:50 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, Jul 3, 2014 at 4:29 PM, Arun Ranganathan a...@mozilla.com wrote: OK, this is fixable. I’ll ensure that the read operation’s synchronous component does return the results thus far, but that FIleReaderSync itself ignores them in case of a midstream error, unless we collectively agree that it SHOULD return partial instead of “all or nothing” as an API. My understanding of the FileReaderSync requirement is all or nothing, but I’m open to being wrong via bug filing. That would mean you would get different results between using FileReaderSync and XMLHttpRequest. That does not seem ideal The implementation train has already left the station on this. The movitation of an “ideal match-up with XMLHttpRequest doesn’t seem strong enough to revisit this by filing browser bugs across implementations (but Cc’ing K. Huey also). We agreed some time ago to not have partial data. The tasks are still a bit of a concern as a normal implementation would not queue tasks. E.g. it's not even clear what task loop Fetch would retrieve this from as fetch is itself a different process. Then let’s have a different read operation that doesn’t use the FileReader Task Source. The primary API in File API uses the event loop (FileReader), so for that purpose, the existing read operation is important. Early discussions about this made me feel that a task queue based system to asynchronously read blobs could be reusable by Fetch, and by future Promise-based APIs. Since it’s not reusable for Fetch, let’s not try and force it. We’ll do a new one along the lines you described at the start of this email. — A*
Re: File API: reading a Blob
On Jul 3, 2014, at 4:14 AM, Anne van Kesteren ann...@annevk.nl wrote: It's unclear to me why we'd want to use the event loop for this, basically. The FileReader object uses the event loop; your initial request for Fetch was to have a reusable “abstract” read operation which used tasks. You’ve since changed your mind, which is totally fine: we could have a different read operation that doesn’t use the event loop that’s put in place for Fetch, but FileReader needs the event loop. We’re talking about an abstract model in specification land on which to pin higher level concepts that culminate eventually in JS objects. It’s useful (I agreed long ago), but overhauling the current read operation for a change of mind/model is a lot of pain without elegant gain. Also, since there isn’t an observable stream or an object, but merely things like load progression (JPEG progression), tasks does seem useful for that. The one thing I suggested which we could do better is the “one byte vs. 50ms” model, and use the “chunk” concept of bytes that the streams folks use. The one thing I’m not clear on here is how to get to a pull-chunk size for files, but I think we could do this better. Also, the current set of synchronous steps (which I could opt not to use, granted, but other APIs already might, and I'd like us to be consistent) simply return failure when something bad happens rather than returning the bytes read so far. It seems like that is a problem, perhaps following from not having this lower-level description. OK, this could be a problem. But this is really immediately usable by the FileReaderSync object on threads, for which a use case for partial data didn’t materialize (and in general, the spec. shunned partial data — references to those threads date way back in time, but they’re there). It seems that for a synchronous block of file i/o, all or nothing catered to most use cases. But if it IS a problem — that is, if you think synchronous I/O has implications outside of FileReaderSync, OR that FileReaderSync’s return itself should be partial if a failure occurs, then let’s file bugs and solve them. I’d REALLY like to have solid abstract models in place for Fetch, since I buy into the cross-purposability of it. But I’d also like shipping implementations to be defined (99% done in File API), with the small delta remaining — Blob URL autorevoking and Blob closing — to be nailed down. — A*
Re: File API: reading a Blob
On Jul 3, 2014, at 10:17 AM, Anne van Kesteren ann...@annevk.nl wrote: So most of Fetch is asynchronous. If it's invoked with the synchronous flag set it's just that it waits for the entire response before returning. That's why I'd like to use the asynchronous path of reading a blob. But I'd like that not to be observably different from using the synchronous path of reading a blob. That seems wrong. OK, this is fixable. I’ll ensure that the read operation’s synchronous component does return the results thus far, but that FIleReaderSync itself ignores them in case of a midstream error, unless we collectively agree that it SHOULD return partial instead of “all or nothing” as an API. My understanding of the FileReaderSync requirement is all or nothing, but I’m open to being wrong via bug filing. Are you agreed (as far as it is possible for you to agree with me on anything) that the event loop async read might allow us to address cases like JPEG progression? It seems imminently usable here. — A*
Re: File API: reading a Blob
On Jul 2, 2014, at 10:28 AM, Anne van Kesteren ann...@annevk.nl wrote: So what I need is something like this: To read a Blob object /blob/, run these steps: 1. Let /s/ be a new body. [FETCH] 2. Return /s/, but continue running these steps asynchronously. 3. While /blob/'s underlying data stream is not closed, run these substeps: 1. Let /bytes/ be the result of reading a chunk from /blob/'s underlying data. 2. If /bytes/ is not failure, push /bytes/ to /s/ and set /s/'s transmitted to /bytes/'s length. 3. Otherwise, signal some kind of error to /s/ and terminate these steps. Are you sure this simply cannot be done with the existing read operation, which uses annotated tasks for asynchronous use? The existing read operation serves FileReader and FileReaderSync (and maybe other APIs in FileSystem) and hopefully Fetch too. For instance, I thought the idea was that within Fetch to read /blob/ we’d do something like: 1. Let /s/ be a new body. Return /s/ and perform the rest of the steps async. 2. Perform a read operation [File API] on /blob/. 3. To process read… 4. To process read data, transfer each byte read to /s/ and set /s/’s transmitted to the number of bytes read. // Chunked byte transfer is possible within the 50ms delta for process read data. We could specify that here better.// 5. To process read EOF ... 6. Otherwise, to process read error with a failure reason on /s/ …. Why is something like that unworkable and why do we need another variant of the read operation exactly? — A*
Re: File API: Blob URL origin
On Jun 30, 2014, at 7:13 PM, Glenn Maynard gl...@zewt.org wrote: Why would the identifier not just be the blob URL itself? The spec currently makes the identifier just the scheme data, which seems much more complex than it needs to be. revokeObjectURL should simply be Remove the entry from the Blob URL Store for URL. If we want to allow revoking URLs that have a fragment attached it'd still need to strip it off; Firefox does this, but Chrome doesn’t. There’s no good reason not to do this. The scheme data identifier is a hold-over from the “only UUID” model that Fx has right now (the original intent was to have a table full of UUIDs and blob references). I’m happy to change to this model. Also, both Chrome and Firefox treat the entire URL as case-sensitive, eg. Blob:... won't revoke the URL, or uppercasing the hostname portion in Chrome. Using the whole URL as the identifier makes this easy to do. Subsequent attempts to dereference url must return a network error should be removed. That should already be the consequence of unregistering the URL, so this is a redundant requirement. OK, this is a good nit too. — A*
Re: File API: Blob URL origin
On Jul 1, 2014, at 2:32 AM, Anne van Kesteren ann...@annevk.nl wrote: That works for me. That way we can make this a more generic store if god forbid we get more of these schemes. While I think mediastream URLs may have died on the vine, using the same store for filesystem URLs would be good. — A*
Re: File API: Blob URL origin
On Jun 28, 2014, at 4:42 AM, Anne van Kesteren ann...@annevk.nl wrote: I now defined the origin for blob URLs: http://url.spec.whatwg.org/#concept-url-origin Sorry for the delay. Still need to work out the correct Fetch integration. Thanks :) Removed origin extraction from FIle API, but added identifier extraction (based on the same model — that is, running the basic URL parser). This makes adding entires to the Blob URL Store clearer. I think the Blob pieces for Fetch are in place now. http://dev.w3.org/2006/webapi/FileAPI — A*
Re: File API: Blob URL origin
On Jun 30, 2014, at 4:20 PM, Anne van Kesteren ann...@annevk.nl wrote: I don't really understand this. Entries should be added when a blob URL is created. They are! That is, at the time the method URL.createObjectURL(blob) is called on blob, that method adds an entry to the Blob URL Store: http://dev.w3.org/2006/webapi/FileAPI/#add-an-entry I’ve only defined identifier extraction for use with adding an entry. Is that wrong? And should be checked/removed when a blob URL is used (through parsing it in the URL parser, as defined by the URL parser). Yes, this is when URL parser *checks* the Blob URL Store, *after* entries have either been added or are not present (which generates a network error). There should be no need for such a definition in the File API specification. Since adding scheme data as an identifer to the Blob URL Store is the requirement, we only use the basic URL parser to get the scheme data from a generated URL to add to the Blob URL Store. — A*
Re: File API: Blob URL origin
On Jun 30, 2014, at 4:57 PM, Anne van Kesteren ann...@annevk.nl wrote: On Mon, Jun 30, 2014 at 10:48 PM, Arun Ranganathan a...@mozilla.com wrote: They are! That is, at the time the method URL.createObjectURL(blob) is called on blob, that method adds an entry to the Blob URL Store: http://dev.w3.org/2006/webapi/FileAPI/#add-an-entry I’ve only defined identifier extraction for use with adding an entry. Is that wrong? It seems like you could define identifier creation, use that, use the return value to add an entry, and then return blob: + the return value. Creating a URL first and then parsing it again to extract something seems needlessly complicated. Well, the best way to define URL.revokeObjectURL(blobURL) seemed to be in terms of parsing to extract identifier (scheme data) and then delete the entry corresponding to identifier. But you’re absolutely right that URL.create* methods shouldn’t have a dependency on the basic URL parser, and so I’ve redefined those methods along the lines you say above (namely, defining identifier creation, and then Blob URL creation). That’s http://dev.w3.org/2006/webapi/FileAPI/#creating-revoking in today’s editor’s draft. (So *now* maybe the path is clear for Fetch with Blobs.) — A*
Re: Blob URL Origin
On Jun 10, 2014, at 2:57 AM, Anne van Kesteren ann...@annevk.nl wrote: On Tue, Jun 10, 2014 at 12:16 AM, Arun Ranganathan a...@mozilla.com wrote: Right now, the Blob URL Store is defined in terms of units of similar-origin browsing contexts; each unit is required to have a Blob URL Store. As you point out, that allows all origins within document.domain access to a given Blob URL Store. Yeah, so unlike what the discussion claimed thus far, we did not in fact allow that much cross-origin blob URL usage. Only origins within the document.domain reach. 1. Require that entries in the Blob URL Store also store origin I thought this was the idea. The identifier would be http://someorigin:70/uuid”. Yes; there was some discussion about tuples vs. strings on IRC, but I think one leads to the other, and we can define how to extract the origin from a parsed Blob URL in terms of another use of the URL Parser instead of string parsing. 2. Define it strictly as a same-origin store. I’m a bit fuzzy on how exactly to define this; for instance, strictly the origin and not the effective script origin of a Document? We could say that the store is bound to a global object. And then both URL.createObjectURL() and places that parse URLs hook into the entry setting object's global object's blob URL store. At that point the only benefit of putting the origin into the URL is so that new URL(blob).origin works. This seems right; I think it would be rare that a developer would need to check origin, but it’s been pointed out that there some use cases for that. It seems better to introduce a method that doesn’t require creating a new object, but I don’t feel strongly about it. Something that is still unclear to me is what happens when you navigate to a blob URL. I guess that still technically works as the URL parsing would happen within the correct global. If URL parsing doesn’t occur within the correct global, a network error will be the result, since there won’t be a corresponding entry in the Blob URL store that matches the identifier. So I think this sounds workable. — A*
Re: Blob URL Origin
On Jun 9, 2014, at 3:23 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, May 29, 2014 at 11:42 AM, Anne van Kesteren ann...@annevk.nl wrote: However, I wonder if this at a standards level should come into play in the URL parser. After all that creates a structured clone of the blob in question. The lookup for the blob ID should probably fail at that point meaning it does not really matter when you then try to fetch that URL as it will simply not have an associated blob. I filed a bug https://www.w3.org/Bugs/Public/show_bug.cgi?id=25987 for this, but it seems worth discussing here. A blob URL store is already limited to all the origins that can reach each other through document.domain. So cross-origin blob usage was already limited per the specification. It seems like what we should do is instead make this a same-origin store. And then when URLs are parsed you'd only have access to the same-origin (and *not* effective origin) blob URL store. In turn that means it does not matter much whether you put origins in the blob URLs, but I suppose we cold do it for clarity. It would also make new URL(blobURL).origin work. Right now, the Blob URL Store is defined in terms of units of similar-origin browsing contexts; each unit is required to have a Blob URL Store. As you point out, that allows all origins within document.domain access to a given Blob URL Store. We could: 1. Require that entries in the Blob URL Store also store origin (namely the effective script origin of the settings object when the URL was created), in addition to identifier and reference to the Blob object. Requests without same origin as that of the entry in the Blob URL store for that identifier must fail. This keeps the Blob URL Store requirement as is, but adds origin as something in the store. 2. Define it strictly as a same-origin store. I’m a bit fuzzy on how exactly to define this; for instance, strictly the origin and not the effective script origin of a Document? — A*
File API For Review | was Re: Blob URL Origin
The 2 June 2014 Editor’s Draft of the File API solves some bugs and technical issues with Blob URLs. Review is encouraged, with a view towards a LCWD publication: http://dev.w3.org/2006/webapi/FileAPI/ In particular: 1. It nails down syntax differences between user agents on Blob URLs. 2. It specifies origin and origin extraction from Blob URL strings. 3. It provides other pieces of plumbing that specifications like URL and Fetch can use. A few issues remain, but all of these have dependencies: https://www.w3.org/Bugs/Public/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDemail1=arun%40mozilla.comemailassigned_to1=1emailreporter1=1emailtype1=exactlist_id=38706 Notably, dependencies on Fetch (for Fetch of Blob URLs), and dependencies on WebIDL (e.g. removing DOMError altogether requires something done for error handling; supplanting FileList with array requires this part nailed down). I’m happy to have provisional specification text in place before WebIDL adopts fixes, but with guidance from the editor(s) of WebIDL. I expect a CfC to follow about the specification. — A* On May 29, 2014, at 5:42 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, May 29, 2014 at 8:38 AM, Jonas Sicking jo...@sicking.cc wrote: On Thu, May 22, 2014 at 1:29 AM, Anne van Kesteren ann...@annevk.nl wrote: For fetching blob URLs (and prolly filesystem and indexeddb) we effectively act as if the request's mode was same-origin. Allowing tainted cross-origin requests would complicate UUID (for the UA) and memory (for the page) management in a multiprocess environment. Hmm.. I think that is effectively it yes. I.e. even though img says that it wants to permit cross-origin loads, we'd override that if the fetch is for a blob: URL and only permit same-origin loads. Is that what you mean? Yes. However, I wonder if this at a standards level should come into play in the URL parser. After all that creates a structured clone of the blob in question. The lookup for the blob ID should probably fail at that point meaning it does not really matter when you then try to fetch that URL as it will simply not have an associated blob. -- http://annevankesteren.nl/
Re: File API - Writer suspension
On Jun 1, 2014, at 1:22 PM, Julian Ladbury julian.ladb...@berrick-computing.co.uk wrote: I fail to understand why work on this API has been suspended. Just to be clear, by “this API” I think you mean: http://dev.w3.org/2009/dap/file-system/file-writer.html HTML5, JavaScript and CSS together are becoming a natural platform of choice on which to write portable applications. Indeed, I have just started work on just such a project, welcoming the chance it gives to break away from proprietary solutions. An essential part of any mature application is the ability to share data with other applications. As an example, JSON provides an ideal way to do this: but to be useful, it has to be possible to save a JSON file on a local system (internet access cannot, and should not, be taken for granted) which can be transmitted by email or other simple means. I think this is the primary family of use cases around the FileSystem API: http://w3c.github.io/filesystem-api/Overview.html which is a successor specification. A few things: 1. You can already create Blobs without using the BlobBuilder API (which has been deprecated), and you can already save them, but through a user prompt (the “File Save As” dialog). This Specifiction discourse thread is useful for other workarounds, and a discussion of their shortcomings: http://discourse.specifiction.org/t/saving-files-api/63 Yes, this area of the platform is currently underspecified. Hopefully not for long though :) 2. In the interim, you can probably use IndexedDB to address the immediate use case. — A*
Re: Blob URL Origin
On May 22, 2014, at 4:29 AM, Anne van Kesteren ann...@annevk.nl wrote: Thanks, I'm convinced. So now I'd like to know what policy we want so we can carefully define it. The lastest editor’s draft of the File API specifies what we discussed in this email thread as syntax for Blob URLs: http://dev.w3.org/2006/webapi/FileAPI/#DefinitionOfScheme and origin, including how to serialize the Blob URL. For blob URLs (and prolly filesystem and indexeddb) we put the origin in the URL and define a way to extract it again so new URL(blob).origin does the right thing. I wonder if .origin should be static? For fetching blob URLs (and prolly filesystem and indexeddb) we effectively act as if the request's mode was same-origin. Allowing tainted cross-origin requests would complicate UUID (for the UA) and memory (for the page) management in a multiprocess environment. We’re not allowing them. — A*
Re: Blob URL Origin
On May 19, 2014, at 5:33 AM, Frederik Braun fbr...@mozilla.com wrote: On 15.05.2014 22:46, Glenn Maynard wrote: On Thu, May 15, 2014 at 12:07 PM, Jonas Sicking jo...@sicking.cc mailto:jo...@sicking.cc wrote: On Thu, May 15, 2014 at 6:52 AM, Anne van Kesteren ann...@annevk.nl mailto:ann...@annevk.nl wrote: I was thinking about the latter and that would not work if the URL was revoked. Unless we store origin at parse time. Good point. Without using the explicit syntax we couldn't return a consistent result for the origin. I'm not against the explicit model, but I think it would be very desirable if there was a way to get the origin out of the blob URL with some sort of API. It's not unlikely that developers will implement their own (flawed) URL parsing to determine the origin of a blob URL, if it's explicitly in the blob URL. I think this could be done with the existing URL API’s origin attribute so that it does the right thing with a Blob URL: http://url.spec.whatwg.org/#dom-url-origin Of course, right now that won’t work in Fx. I wonder if we could fix that? Or add another static function to URL. —A*
Re: Blob URL Origin
On May 12, 2014, at 8:28 AM, Anne Van Kesteren ann...@annevk.nl wrote: It still seems a bit sad though to tie these URLs to origins in this fashion. Jonas is correct that there are inconsistencies in how data URLs and origins behave across browsers, but it seems like we should sort those out first then if we want a consistent story. Since Blobs can be passed around in a number of well-known ways, it seems that the most legitimate origin of a Blob URL is the origin of the script that coined it. I’m not entirely sure how to take action on “it still seems a bit sad” though. Sad because of security considerations? After drying my tears, I can’t construct a meaningful attack, but I’d welcome more information about what benefits are gained by encoding certain “HTTP-reserved” components of URL nomenclature (and here, Chrome is inconsistent between blob: and filesystem:). Sad because of aesthetics? It’s pretty enough for Safari. And really, all user agents seem to agree that the origin is that of the settings object today. That model seems to work. The remaining question is the pro and con of denoting this in the URL’s syntax. abarth’s advice is to put the syntax horse in front of the origin cart: http://krijnhoetmer.nl/irc-logs/whatwg/20140508#l-913 Also, if it’s “sad” because it doesn’t match data: URL’s way of reckoning origin, that doesn’t seem sad to me. — A*
Re: Blob URL Origin
On May 12, 2014, at 10:31 AM, Boris Zbarsky bzbar...@mit.edu wrote: On 5/12/14, 5:28 AM, Anne van Kesteren wrote: so blob:https://origin:42/uuid would be fine. I'd really rather we didn't make web pages parse these strings to get the origin. A static method on Blob that takes a valid blob: URI and returns its origin seems like it should be pretty easy for UAs to implement, though. We actually aren’t obliging web pages parse these strings to get the origin. In fact, blob: URL strings shouldn’t even be of interest to web pages. They aren’t today, and I don’t envision them being of interest even with “origin tagging.” That is, I can’t think of why exactly a web developer would want to look into the blob: URL strings. UA’s should just “do the right thing” once a Blob URL is coined. The question is really whether origin should be implicit or explicit. Fx’s implementation makes it implicit. so that there’s no way to deduce origins from the Blob URL itself, but it just “does the right thing” in terms of origin strictures. That hasn’t been a problem, but it’s hard to spec. it that way. Also, it makes Blob URLs only usable within APIs that are aware of them, which honestly is the case today. So what if we tag origin into the strings? Would that be so bad? It’s not doing anything other than denoting the incumbent script setting object’s origin, no? Even in the Chrome/Safari cases, I can’t think of web developers using that information. — A*
Re: data:, was: Blob URL Origin
On May 12, 2014, at 6:26 AM, Julian Reschke julian.resc...@gmx.de wrote: Could you please clarify what spec you are referring to, and in which way it's not implemented correctly? Well, I think http://tools.ietf.org/html/rfc6454#section-4 is is supposed to be normative for data: URL origin. But, implementations don’t behave this way; the problem is inheriting origins. Here’s the pertinent thread: http://lists.w3.org/Archives/Public/public-webapps/2014JanMar/0682.html
Re: [April2014Meeting] Building an Issue and Bug focused agenda
I took the liberty of adding File + FileSystem to Thursday's agenda, taking place at 11.30 AM PT. I am happy to dial in on either day, so this isn’t set in stone if someone strongly objects. — A* On Apr 9, 2014, at 11:13 AM, Arthur Barstow art.bars...@nokia.com wrote: On 4/9/14 12:03 AM, ext Maciej Stachowiak wrote: Would the WG be willing to put Web Components related topics on Friday when building the agenda? (Note: there are at least some Apple folks who will be there both days in any case.) On Apr 8, 2014, at 8:40 AM, Dimitri Glazkov dglaz...@chromium.org mailto:dglaz...@chromium.org wrote: Actually, Friday sounds better for me too! For a relatively large subject area (3 specs and the various discussion topics Dimitri identified [1]) and broad interest, it could be better to start discussions on Thursday and try to narrow the more contentious topics (and potentially benefit from discussions during that evening's gatherings and/or a good night's sleep). On the other hand, having a long-ish dedicated time slot on Friday also has benefits. Having seen no objections to putting Web Components on Friday's agenda, I just added it to Friday's 09:00 - 12:30 time slots [2]. -AB [1] http://lists.w3.org/Archives/Public/public-webapps/2014AprJun/0074.html [2] https://www.w3.org/wiki/Webapps/April2014Meeting#Agenda_Friday_April_11
Re: [April2014Meeting] Building an Issue and Bug focused agenda
On Apr 2, 2014, at 6:50 AM, Arthur Barstow art.bars...@nokia.com wrote: File API; led by Jonas (will Arun join remotely?); which high priority bugs need discussion; status of LC processing; next steps If a dial-in is provided, I’d be delighted to join remotely. — A*
Re: On starting WebWorkers with blob: URLs...
On Mar 12, 2014, at 6:54 PM, Ian Hickson wrote: For blob: URLs we agreed to make this pretty explicit: http://dev.w3.org/2006/webapi/FileAPI/#originOfBlobURL Unfortunately, scripts don't have origins these days, so this definition doesn't really work. It didn't work since it wasn't formally specific about effective script origin according to the script setting object, but I've fixed that so hopefully this definition should work: http://dev.w3.org/2006/webapi/FileAPI/#originOfBlobURL -- A*
Re: On starting WebWorkers with blob: URLs...
On Feb 19, 2014, at 7:09 PM, Jonas Sicking wrote: On Wed, Feb 19, 2014 at 3:51 PM, Travis Leithead travis.leith...@microsoft.com wrote: Seems like our specs are getting a little behind implementations. In IE11 we are finding that several new sites, notably those using WebGL content, have a dependency on starting web workers with a Blob URL. As I understand it: The W3C Web Workers spec (CR stage) forbids use of the data and [implicitly] blob protocols The WHATWG HTML spec (Living stage) allows data protocol, but not blob. Implementations of Firefox and Chrome support blob protocols 1. Seems like a spec should say this somewhere... Agreed! It's a bit tricky since the concept of origins and thus same origin for data: and blob: is a bit unclear still. I.e. browsers don't behave consistently. Definitely not between each other, and sometimes not internally within a browser IIRC. Well, for data: URLs what's normative might be: http://tools.ietf.org/html/rfc6454#section-4 but sadly that's not really clear (what's host here?). For blob: URLs we agreed to make this pretty explicit: http://dev.w3.org/2006/webapi/FileAPI/#originOfBlobURL Also, Blob URL Stores apply to each unit of similar origin browing contexts, which means that URL.createObjectURL and URL.createFor essentially give URL fetch and URL parse license to operate only within the origin of the script those methods apply to; URL.revokeObjectURL only applies to that origin also. So now, we've made it pretty explicit for Blob URLs. 2. In the W3C where would we spec this? (Workers V2?) I care less strongly about this. There's also the synchronous message passing API which I'd still like to see added to the workers spec. IMHO it makes sense in Workers V2. -- A*
Re: [File System APIs] If one is good, then two must be better?
On Feb 5, 2014, at 7:03 AM, Charles McCathie Nevile wrote: Similarly the issue here is not whether we can make a specification for one or the other approach that *could* be a standard, since it seems we can, but whether one or the other is a clear candidate to be a real standard - i.e. what people *will* actually do... I think one of the mistakes with IndexedDB (and appcache for that matter) was that the participants in the discussion were too heavily biased toward browser implementors, without enough input or involvement from working developers. Which meant that we standardised something that didn't meet people's expectations as well as we and they hoped. Of course I agree that the path to standardization should involve input from developers, and should be based on what people will actually do. There's been some support for a Promises-based FileSystem API in lieu of a callback-based one, both on this listserv and on public-script-coord, which is where the proposal started: http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html To borrow an expression from that thread, we're not blank-slateists here: there are 3 separate APIs for JavaScript file system APIs, including the one in node.js (implemented as a posix-style API, familiar from other programming environments), from Google (implemented in Chrome as a callbacks-based asynchronous API), and from Mozilla (implementing the proposal we're discussing here [1], but sandboxed as of now). This is a call for feedback, from web developers and implementers. I only made the point that when considering *implementations* we should consider similar beasts. The thread about having a FileSystem API is a long one, and goes back a long time. The motivation is to provide something asynchronous (but not callbacks stapled to posix shells), in keeping with the flavor of APIs currently deployed for the web (Promises, not events or callbacks), that is easy to use and with a low probability of doing bad things. Feedback from web developers about that endeavor is invaluable. I hope that when we make a choice it's one that not only matches the reality of what gets implemented, but helps us provide what the market really wants. I presume everyone here hopes for that. I think a key part of how to get there is to listen carefully to the feedback we get, and look around at what people are doing, rather than just relying on formulaic rules of thumb or bureaucratic tallying of test results. I agree with the spirit behind this comment, and am optimistic that you don't actually think that the call for opinions on a proposal, or the call for implementor support, is a sign of a formulaic rule of thumb or a bureaucratic tallying of test results. -- A* [1] Interfaces only: http://w3c.github.io/filesystem-api/Overview.html
Re: [File System APIs] If one is good, then two must be better?
On Feb 5, 2014, at 11:12 AM, Ian Clelland wrote: The great thing about Cordova is that it doesn't have to be a single platform -- every developer has the power to choose the APIs that they want to use, and every published app is essentially running on its own custom web platform. I wasn't presenting the Cordova data point as we're doing this already, and it's going to be too much work to change now. Far from it; Cordova is a flexible platform, and someone could write a plugin providing the Mozilla-backed Filesystem API tomorrow, and it wouldn't cause any hardship to developers who didn't choose to adopt it. My point was merely to say that every published app running on its own custom web platform is a different kind of implementation than a web browser, that's all :-) I think it's *really great* that Cordova provides a flexible platform for web APIs to land, and that plugins can be deployed rapidly. I'd *gladly* welcome an implementation of the FileSystem proposal in Cordova, or at least more feedback on the proposal as it stands currently. -- A*
Re: [File System APIs] If one is good, then two must be better?
On Feb 5, 2014, at 6:51 AM, Sebastian Kippe wrote: For what it’s worth, we’re currently building a mobile app for use with both Cordova and Web browsers. The main use case of our app is caching large video files and playing them back directly from the filesystem. So filesystem URLs are a critical feature there. Other than that, the callback-based approach of the Google proposal let us end up in callback hell right from the start, so we wrapped the whole API in promises as best as we could, just to make it usable for our app developers. In that department, Mozilla’s promises-based proposal might be much cleaner and easier to work with. This is really useful feedback :-) Specifics about the API as it develops would be really helpful as well. -- A*
Re: [File System APIs] If one is good, then two must be better?
On Feb 4, 2014, at 6:15 AM, Charles McCathie Nevile wrote: On Mon, 03 Feb 2014 19:09:53 +0400, Arthur Barstow art.bars...@nokia.com wrote: On 1/31/14 10:44 AM, ext Ian Clelland wrote: Hi Art, For what it's worth, theFile API: Directories and System is also implemented (and supported) by Apache Cordova[1]. The implementation is essentially complete for mobile applications on Android, iOS and FireOS, with nearly-complete support on Blackberry and Windows Phone. While our plugin registry was counting downloads, it was the most-downloaded plugin for the platform by a wide margin, so I believe it is being used actively. Thanks for this information Ian! I don't know if Cordova should count as a browser implementation for the purposes of this WG, but we are implementing the APIs and making them available to (hybrid) web application developers. The group has some flexibility regarding the specifics of the interoperability criteria used to advance a spec along the Recommendation track, but we haven't talked about the criteria for these specs since they are still working drafts. And the particular question here isn't about CR criteria, but about whether one or other approach is more likely to achieve the consensus of interoperable implementation. Which essentially means whether implementations are likely to switch, or credible future implementors have a strong preference for one over the other. In which case, what Cordova does (and more to the point what developers do with it) seems relevant information to consider as we try to find a consensus. Two interoperable implementations of a specification should determine the way forward. While I think distributions like PhoneGap are extremely useful as web-like abstractions on top of disparate mobile platforms, it is not straightforward to make a clear apples to apples comparision for API interoperability between PhoneGap and a web browser, or conduct common test cases. Naturally, the distinctions blur, but I still think they exist. A web page using the FileSystem API in JavaScript and working in two separate browser implementations seems like a good measure of interoperability, and I think this should be what helps us make a determination. -- A*
Re: [File System APIs] If one is good, then two must be better?
On Jan 31, 2014, at 10:21 AM, Arthur Barstow wrote: Hi Eric, Arun, Jonas, All, We also have this spec from Arun and Jonas: * FileSystem API http://w3c.github.io/filesystem-api/Overview.html. The last update of the ED was 2-Oct-2013 and this spec has not been published as a TR. My understanding is the only implementation of Eric's APIs is Chrome. I do not know the implementation status of Mozilla's spec. If anyone has additional information about the implementation status or plans of either effort, please let us know. The specification is being implemented in Fx with the interfaces as in the document you link to, but the current implemention in Fx is sandboxed and not exposed to web content. We'd like to change that at some point. The last discussion about the relationship between these different efforts was August 2013 [Aug-2013] and prior to that, there was some discussion during the April 2013 f2f meeting [April-2013]. Ultimately, I think there is broad agreement a single API that is broadly implemented and deployed would be `best` (f.ex. reduces FUD, lightens implementation costs, lightens deployment costs, etc.). Although I would (still) like to be optimistic we can agree to converge on a single API, previous discussions about this do make me skeptical ... Eric, Arun, Jonas - can you agree and commit to converge your efforts, f.ex. just have a single API? We can certainly commit to editing the proposal we put forward, and would like to hear from other browser vendors about their opinions. That will inform what we should do collectively. -- A*
Re: CfC: publish Candidate Recommendation of File API; deadline November 28
On Dec 18, 2013, at 6:38 PM, Arthur Barstow wrote: Are there any other open bugs, issues, comments, etc. for File API? https://www.w3.org/Bugs/Public/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDemail1=arun%40mozilla.comemailassigned_to1=1emailreporter1=1emailtype1=exactlist_id=30639 is the current list. Some came up on 12/15/2013 and are minor. Unless I hear otherwise, I will assume the group wants to address the above before a new Technical Report is published and depending on the outcome of these issues, the next publication could be a Last Call Working Draft or a Candidate Recommendation. I agree with this. -- A*
Re: File API: closed Blob objects
On Dec 12, 2013, at 10:07 AM, Anne van Kesteren wrote: See https://www.w3.org/Bugs/Public/show_bug.cgi?id=24072 for context. What's the use case for close()? Essentially, the use case is for web developers to be able to flush a Blob reference that isn't used, allowing the user agent to manage resources. This was expressed in: https://www.w3.org/Bugs/Public/show_bug.cgi?id=16952#c2 If we are to have close(), let's define it in such a way that once you invoke it the Blob represents the empty byte sequence. This way other code that takes and operates on Blob objects does not have to special case closed Blob objects, but can just treat them as Blob objects that happen to be empty. Currently, we say that : Calling the close method on a Blob must disable it such that it cannot be used again; dereferencing a blob: URL bound to a Blob object on which close() has been called results in a network error. A Blob on which close() has been called must have a size of 0. This must be an irreversible and non-idempotent operation; once close() has been called on a Blob, that Blob cannot be used again. Is size of 0 a sufficient constraint? I'm inclined to think so, and that we already support your suggestion. Also, note that neutering a transferable object and calling close on Blob are *different things.* We can make Blob's size 0 on close(). But, neutering a transferable object should throw on subsequent access. You'll recall that we forced them to be different things based on bugs such as: 1. https://www.w3.org/Bugs/Public/show_bug.cgi?id=18611 and 2. https://www.w3.org/Bugs/Public/show_bug.cgi?id=23416 -- A* -- A*
Re: File API: closed Blob objects
On Dec 12, 2013, at 2:01 PM, Jonas Sicking wrote: On Dec 12, 2013 9:55 AM, Arun Ranganathan a...@mozilla.com wrote:Reading from a Blob that has been closed() is a developer error with high degree of certainty. Nothing useful can come out of that. So I think we do the developer a benefit by treating it as such and signaling an error through whatever mechanism is appropriate depending on how the reading was done. We can certainly debate if throwing an exception or if asynchronously reporting a network error is the right thing to do for things like XHR.send(closedBlob) or FileReader.readAsX(closedBlob). So right now (in spec.) any Blob URL operations on a closed Blob must result in a network error. FileReader.readAsX(closedBlob) operations are equivalent to reading a Blob of size 0. I can understand the argument that the above might be a bit inconsistent; after all, we could do a Blob URL operation on a Blob of size 0 (where nothing happens, since no bytes are fetched), just as we allow FileReader reads on a resource of size 0 (e.g. result is a 0 bytes ArrayBuffer). We should pick one thing: treat it as 0, and proceed accordingly (often, this is silent failure), or treat it as something that should be reported back as an error. If we go the throw or error route, since we've already spec'd a network error for fetched Blob URLs when the underlying Blob has been closed with .close(), the choice of what to do for FileReader.readAsX(closedBlob) is whether to throw or to asynchronously use an error event with a DOMError (possibly with a NotReadableError). On December 12, 2013 at 2:06 PM, bzbarsky wrote: Or form submission after fileInput.files[0].close() ? This example makes me think that on balance, for consistency, maybe we should throw, and so should XHR.send(closedBlob). -- A*
Re: File API: closed Blob objects
On Dec 12, 2013, at 3:14 PM, Jonas Sicking wrote: This does seem like an argument to create a asynchronous error rather than throwing an exception though, since for form submission there would be no place to throw an exception from. Just to be clear, are you proposing an asynchronous error reported through an error event firing off of FileReader with a DOMError? Also, it isn't clear what fileListObject.files[0].close() would do for an asynchronous error, so perhaps both are necessary: treating it as 0 bytes (empty byte sequence) *and* (where possible) using an asynchronous error reporting mechanism. -- A*
Re: CfC: publish Candidate Recommendation of File API; deadline November 28
On Dec 1, 2013, at 10:24 AM, Arthur Barstow wrote: Since this CfC was started, Arun reopened [23853] on November 28 and [23946] was filed on November 30. Arun - what's the plan here vis-à-vis this CfC? -Thanks, ArtB [23853] https://www.w3.org/Bugs/Public/show_bug.cgi?id=23853 Timely implementor feedback will help close this one out. I will initiate a separate email thread about this issue. [23946] https://www.w3.org/Bugs/Public/show_bug.cgi?id=23946 I don't believe this issue is accompanied with a strong enough use case, and think it likely that it will be marked WONTFIX during the timeframe of the CfC. -- A*
Re: [FileAPI] LC Comment Tracking
Art, All LC commentary (http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912) has been addressed and I think the draft is ready to be published as CR status. The draft is: http://dev.w3.org/2006/webapi/FileAPI/ -- A* On Nov 8, 2013, at 10:15 AM, Arun Ranganathan wrote: Hi Art, On Nov 7, 2013, at 9:40 AM, Arthur Barstow wrote: Since it appears you will not be at WebApps' f2f meeting next week, I would appreciate it if you would please summarize the status of the comment processing, your next steps, etc. I am especially interested in whether or not you consider any of the bug fixes you applied as substantive and/or add a new feature (which would require a new LC). Most LC commentary that was substantive became a spec bug; I've fixed most such spec. bugs, and the contributor/commentor has been notified. In my opinion, the biggest change is to the File constructor. This is https://www.w3.org/Bugs/Public/show_bug.cgi?id=23479. I don't think this is a new feature, since the previous document pushed to /TR had a constructor, athough a different signature. Other changes include moving Blob URL to be redefined in terms of terminology in the WHATWG URL spec, in lieu of ABNFs. If you provide a dial-in on the day that you discuss File + FileSystem, I can try and dial in, but this depends on time. There will be others present from Mozilla :) The LC commentary is tracked at http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912 -- A* -Thanks, ArtB [1] http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912 On 9/12/13 10:39 AM, ext Arthur Barstow wrote: [ Bcc public-sysapps ; comments from SysApps are welcome ] This is a Request for Comments for the 12 September 2013 Last Call Working Draft of File API: http://www.w3.org/TR/2013/WD-FileAPI-20130912/ The comment deadline is October 24 and all comments should be sent to the public-webapps@w3.org list with a subject: prefix of [FileAPI]. The spec's bug list is [Bugs] and the few `approved` tests we have can be run in a browser at [Tests]. -Thanks, ArtB [Bugs] http://tinyurl.com/Bugs-FileAPI [Tests] http://w3c-test.org/web-platform-tests/master/FileAPI/
Re: Why no TypeError in FileReaderSync.readAsArrayBuffer
Norman, On Nov 7, 2013, at 5:00 PM, Norman Rzepka | scalable minds wrote: because I didn't know a FileEntrySync was actually not a File, I passed an instance of FileEntrySync to a FileReaderSync. Instead I got a NotFoundError which I found very confusing as I know the file was present. So I was wondering why there is no TypeError for such situations? The API you're testing -- namely code which uses FileEntrySync primitives -- is likely to work in Chrome only, and isn't a standard with other browser vendors, which do implement the File API. Currently, the FileSystem API (which, going forward is this draft; http://w3c.github.io/filesystem-api/Overview.html) doesn't have a FileEntrySync primitive. And the File API is http://dev.w3.org/2006/webapi/FileAPI/#FileReaderSync . In this particular case, I think it's the right behavior: FileReaderSync expects a valid File or Blob reference, and isn't able to identify one, so says one isn't found. -- A* Best, Norman -- Norman Rzepka norman.rze...@scalableminds.com http://scalableminds.com scalable minds UG (haftungsbeschränkt) Co. KG Amtsgericht Potsdam, HRA 5753 Geschäftsführer: Dominic Bräunlein, Tom Bocklisch, Tom Herold, Norman Rzepka, Thomas Werkmeister
Re: Why no TypeError in FileReaderSync.readAsArrayBuffer
On Nov 11, 2013, at 11:08 AM, Tab Atkins Jr. wrote: On Mon, Nov 11, 2013 at 5:57 AM, Arun Ranganathan a...@mozilla.com wrote: In this particular case, I think it's the right behavior: FileReaderSync expects a valid File or Blob reference, and isn't able to identify one, so says one isn't found. No, NotFoundError is for when some search is unsuccessful but necessary. Looking at the type of an argument doesn't seem like a search without a *lot* of stretching. It's a TypeError, if you're going to throw anything. You're right, a TypeError is less of a stretch. This should be a browser bug, though. -- A* ~TJ
Re: [FileAPI] LC Comment Tracking
Hi Art, On Nov 7, 2013, at 9:40 AM, Arthur Barstow wrote: Since it appears you will not be at WebApps' f2f meeting next week, I would appreciate it if you would please summarize the status of the comment processing, your next steps, etc. I am especially interested in whether or not you consider any of the bug fixes you applied as substantive and/or add a new feature (which would require a new LC). Most LC commentary that was substantive became a spec bug; I've fixed most such spec. bugs, and the contributor/commentor has been notified. In my opinion, the biggest change is to the File constructor. This is https://www.w3.org/Bugs/Public/show_bug.cgi?id=23479. I don't think this is a new feature, since the previous document pushed to /TR had a constructor, athough a different signature. Other changes include moving Blob URL to be redefined in terms of terminology in the WHATWG URL spec, in lieu of ABNFs. If you provide a dial-in on the day that you discuss File + FileSystem, I can try and dial in, but this depends on time. There will be others present from Mozilla :) The LC commentary is tracked at http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912 -- A* -Thanks, ArtB [1] http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912 On 9/12/13 10:39 AM, ext Arthur Barstow wrote: [ Bcc public-sysapps ; comments from SysApps are welcome ] This is a Request for Comments for the 12 September 2013 Last Call Working Draft of File API: http://www.w3.org/TR/2013/WD-FileAPI-20130912/ The comment deadline is October 24 and all comments should be sent to the public-webapps@w3.org list with a subject: prefix of [FileAPI]. The spec's bug list is [Bugs] and the few `approved` tests we have can be run in a browser at [Tests]. -Thanks, ArtB [Bugs] http://tinyurl.com/Bugs-FileAPI [Tests] http://w3c-test.org/web-platform-tests/master/FileAPI/
Re: [FileAPI]
Note that all LC Commentary, including that sent on this listserv, is tracked here: http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912 -- A* On Oct 31, 2013, at 9:12 AM, Arthur Barstow wrote: Hi, For the purposes of tracking your comments for the September 12 File API Last Call Working Draft, please let us know if Arun's reply is satisfactory or not. In the absence of a reply from you by November 7, we will assume Arun's reply is OK with you. -Thanks, ArtB On 10/23/13 6:04 PM, ext Arun Ranganathan wrote: Hi there! On Oct 23, 2013, at 12:32 PM, psweatte wrote: 7.2 Interface File: -add creationDate property Thanks for your feedback. *Most* filesystems don't really have a creation time. While Windows does, Unix-style OS return the *change time or last modified time. Since we want fidelity with OS filesystems wherever possible, I'm not sure this is a viable property to add. -add size property This already exists via the inheritance relationship with Blob. -If the last modification date and time are not known, the attribute must return an empty string Currently the spec says to return the current date and time. It's a Date not a String. 8.3. Event Handler Attributes I think the current set of event handler attributes is sufficient, especially given the backdrop of event models in general not being the best way to asynchronously access large data sets such as File and Blob. -add onNotfounderror event handler -add onReaderror event handler -add onSecurityerror event handler -add onHTTPerror event handler -add onSelectfrombrowse event handler
Re: [FileAPI]
Hi there! On Oct 23, 2013, at 12:32 PM, psweatte wrote: 7.2 Interface File: -add creationDate property Thanks for your feedback. *Most* filesystems don't really have a creation time. While Windows does, Unix-style OS return the *change time or last modified time. Since we want fidelity with OS filesystems wherever possible, I'm not sure this is a viable property to add. -add size property This already exists via the inheritance relationship with Blob. -If the last modification date and time are not known, the attribute must return an empty string Currently the spec says to return the current date and time. It's a Date not a String. 8.3. Event Handler Attributes I think the current set of event handler attributes is sufficient, especially given the backdrop of event models in general not being the best way to asynchronously access large data sets such as File and Blob. -add onNotfounderror event handler -add onReaderror event handler -add onSecurityerror event handler -add onHTTPerror event handler -add onSelectfrombrowse event handler
Re: Problem in the Entry interface description (filesystem)
Greetings Alexis, The specification actively edited now is: http://w3c.github.io/filesystem-api/Overview.html -- A* On Oct 10, 2013, at 10:00 AM, alexis dufrenoy wrote: Hello, In the description of the Entry interface located here: http://dev.w3.org/2009/dap/file-system/pub/FileSystem/#the-entry-interface I noticed the following sentence: An abstract interface representing entries in a file system, each of which may be a File or DirectoryEntry., with a link leading to the File interface. Obviously, the File interface is not a descendant of the Entry interface, but FileEntry is, as can been easily checked on both descriptions: http://dev.w3.org/2006/webapi/FileAPI/#dfn-file http://dev.w3.org/2009/dap/file-system/pub/FileSystem/#the-fileentry-interface File is a Blob. I hope it helps, but I'm sure I'm not the first to notice. Regards, Alexis Dufrenoy
Re: [FileAPI] Questions on using Blobs for img src and a href
On Oct 3, 2013, at 6:35 PM, Brian Matthews (brmatthe) wrote: First is the status bar display for anchors while hovering over them. As expected, it's the blob URL. While this is completely correct and exactly what I'd expect, I'm not sure how useful it is. For an anchor with a normal URL, the user is told something about where the resource is (the domain name and path, http://example.com/order;), and what it is (the last element of the path, invoice.pdf). With a blob URL, they're told where it is (at least for those who know what a blob URL is, or accept that blob:... is something magic they don't need to know about :-) ), but nothing about what it is. This is an implementation issue that goes beyond the spec. (as others have remarked, this kind of thing falls under the purview of UI/UX, which web specifications rarely touch on meaningfully), but I think it is probably a useful informative comment that I'll add as part of LC Commentary changes. I think you should file bugs on all browser projects that you'd like to see this kind of thing implemented in. It's a useful feature for end-users. Second, and related, is what happens when someone saves an image or target of a link. In both cases with normal URLs, there's a name component and the user agent can use that as the name of the resulting file, or suggest it if doing a Save As With blob URLs, there is no name, so the user agent has to make up a name. So with Firefox one gets fun names like index.gif (when saving an img), or bf_UK+0O.gif and y+f+wR9a.pdf (when saving the target of an anchor), and Chrome uses the opaqueString part of the URL, resulting in names like 49c122d8-0958-4dfd-ac9c-0a6245c5f91f..gif. None of those exactly brim with semantic content. OK, I've filed https://www.w3.org/Bugs/Public/show_bug.cgi?id=23450 I'm not fully convinced by other commentors on this listserv that we should use Content-Disposition as how to do this. I think how Save As is implemented is a detail left to browsers, but I think the part of using file name is useful during Save As. Also, 8.5.6 step 1 in the spec starts Fire a progress event called error. Set the error attribute;. Doesn't firing an event call the event handlers immediately? If so, this seems to be saying to call the error handler *before* setting the error attribute, which seems backwards. Yes, I agree! https://www.w3.org/Bugs/Public/show_bug.cgi?id=23451 Thanks for your thoughtful LC commentary. -- A*
Re: [file-{writer,system}-api] Seeking status and plans
On Oct 2, 2013, at 12:29 PM, Arthur Barstow wrote: Hi Eric, If any of the data for your two File APIs in [PubStatus] is not accurate, please provide corrections. Also, if you have any new information re your plans for these specs (last published 17-Apr-2012), or the specs' status with respect to being feature complete, please let us know. All - besides Chrome, if there are other implementations of these specs, please let us know. -Thanks, ArtB I've created a draft of the FileSystem API in GitHub, based on discussions with w3-staff: https://github.com/w3c/filesystem-api This API is different than the Google FileSystem API proposal, athough it will refine the concept of the filesystem: scheme, initially proposed by Google. -- A*
Re: CfC: publish LCWD of File API; deadline August 23
The LCWD of File API is http://www.w3.org/TR/FileAPI/ LC comment period is till October 24 2013 -- comments strongly encouraged! -- A* On Aug 16, 2013, at 8:35 AM, Arthur Barstow wrote: Arun proposed (see below) WebApps publish a Last Call Working Draft of File API and this is a Call for Consensus to do so, using the following ED as the basis: http://dev.w3.org/2006/webapi/FileAPI/ This CfC satisfies the group's requirement to record the group's decision to request advancement for this LCWD. Note the Process Document states the following regarding the significance/meaning of a LCWD: [[ http://www.w3.org/2005/10/Process-20051014/tr.html#last-call Purpose: A Working Group's Last Call announcement is a signal that: * the Working Group believes that it has satisfied its relevant technical requirements (e.g., of the charter or requirements document) in the Working Draft; * the Working Group believes that it has satisfied significant dependencies with other groups; * other groups SHOULD review the document to confirm that these dependencies have been satisfied. In general, a Last Call announcement is also a signal that the Working Group is planning to advance the technical report to later maturity levels. ]] The proposed review period for this LC is 4 weeks. There are two open [Bugs] for this spec. If you have any comments or concerns about this CfC, please send them to public-webapps@w3.org by August 23 at the latest. Positive response is preferred and encouraged and silence will be considered as agreement with the proposal. -Thanks, AB [Bugs] http://tinyurl.com/Bugs-FileAPI Original Message Subject: File* Specifications | Status and some Promises Resent-Date: Thu, 15 Aug 2013 17:50:25 + Resent-From: public-webapps@w3.org Date: Thu, 15 Aug 2013 13:49:38 -0400 From: ext Arun Ranganathan a...@mozilla.com To: Web Applications Working Group WG public-webapps@w3.org Greetings WG, There are three moving proposals in the File* arena, and I thought I'd catpure what these are and what the status of these are. 1. The File API, currently updated as editor's draft at http://dev.w3.org/2006/webapi/FileAPI/ Review strongly encouraged :) Notable fixes are: i. A File constructor has been added, which has been a longstanding request. ii. A new static method on URL called URL.createAndRevoke has been added that gets defaults right, and does away with autoRevoke, which nobody implemented for URL.createObjectURL. iii. A Blob URL Store and a Revocation List, useful for URL Fetch (see https://www.w3.org/Bugs/Public/show_bug.cgi?id=17765). This draft should be ready for LC and should be considered the File API v1. 2. Re-writing File APi to be Promises based, which should be considered a v2. The biggest departure from the current state of technology here would really be that we can have Promises right off the Blob itself, which is zeitgeist thinking about these things (and admittedly makes life easier than an XHR-inspired FileReader, which will be made legacy). The best stab at a strawperson was this one by Jonas: http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0727.html But Stream itself temporarily lives here without too much implementation backbone: https://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm I think we *should* have a Promises-based File API off of Blob, but I'm not totally sure about the Stream API as it is currently written; I'm amenable to editing the File API v2 portion, modulo our proposal for Stream. 3. Then there's the FileSystem API, currently discussed here: http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html This was the upshot of discussions on this listserv, as well as internally at Mozilla. I'm amenable to editing this, which is separate but related to 1. and 2. -- A*
Re: File API: File's name property
On Sep 6, 2013, at 11:42 AM, Anne van Kesteren wrote: On Wed, Sep 4, 2013 at 11:45 PM, Glenn Maynard gl...@zewt.org wrote: On Tue, Sep 3, 2013 at 12:04 PM, Anne van Kesteren ann...@annevk.nl wrote: The problem is that once you put it through the URL parser it'll become /. And I suspect given directory APIs and such it'll go through that layer at some point. I don't follow. Backslashes in filenames are escaped in URLs (http://zewt.org/~glenn/test%5Cfile), like all the other things that require escaping. If the raw input to the URL parser includes a backslash, it'll be treated as a forward slash. I am not really expecting people to use encodeURI or such utilities. I think it may be ok to restrict / and \. I don't think we lose too much here by not allowing historically directory delimiting characters in file names. The question is what to do with a / or a \. I'm inclined to say UAs should treat those as U+FFFD. Well, my suggestion was rawName and name (which would have loss of information), per the current zip archive API design. Having a separate field is fine. This is specific to ZIPs, so it feels like it belongs in a ZipFile subclass, not File itself. Is it? There's no other file systems where the file names are effectively byte sequences? If that's the case, maybe that's fine. Well…. Some file systems don't store names as unrestricted byte sequences (older Windows), but GNU systems usually do. Some byte sequences are not valid names. Conversely, names of existing files may not be representable as byte sequences (and sometimes there are two representations -- e.g. Amèlie.txt will either use 00e9 or 0065 0031 for the è -- both are Unicode equivalents, but are different byte sequences). Some file systems perform Unicode canonicalization on file names, which is more or less what I think the Web should do. I think we run only a small risk of information loss, but I DO think that File name should be an [EnforceUTF16] DOMString. That way, we have the best shot at byte sequences based on the underlying characterization. Summary: I'll punt on File.rawName till a rainier day than today, but I will restrict / and \ since they are historically directory separators. I know that there are OTHER characters that we can also restrict, but these two are the big ones and get us some 80-20 sanitization :) Glenn said: It might be better to wait until we have a filesystem API, then piggyback on that... +1. -- A*
Re: File API: File's name property
Well, https://www.w3.org/Bugs/Public/show_bug.cgi?id=23138 is to make the 'type' attribute a ByteString. Is that your request here for the name attribute as well? It wouldn't be wise to restrict '/' or '\' or try to delve too deep into platform land BUT the FileSystem API introduces directory syntax which might make being lax a fly in the ointment for later. On Aug 29, 2013, at 10:48 AM, Anne van Kesteren wrote: As currently specified File's name property seems to be a code unit sequence. In zip archives the resource's path is a byte sequence. I don't really know what popular file systems do. Given that a File has to be transmitted over the wire now and then, including it's name property value, a code unit sequence seems like the wrong type. It would at least lead to information loss which I'm not sure is acceptable if we can prevent it (or at least make it more obvious that it is going on, by doing a transformation early on). We may also want to restrict \ and / to leave room for using these objects in path-based contexts later. -- http://annevankesteren.nl/
Re: File API: lacks model
Anne, This feedback is a bit vague. I think you should describe the underlying model for Blob/File so other specifications can more easily hook into it. E.g. if I want to represent a File object with a name /x/ and type /y/ there's not a clear way to do that right now. This also leads to issues such as [value space of properties] Why isn't there a clear way to do that now? https://www.w3.org/Bugs/Public/show_bug.cgi?id=23138 notwithstanding, you can either obtain a File from the underlying file system with a best guess type by the UA, OR create one with a constructor. What's missing exactly? E.g. A |blob| provides asynchronous access to a byte sequence. A blob has an associated |type| which is a MIME type. (Which could be a link to the variant of MIME type you use.) And then you build the API on top of this model. I treat http://mimesniff.spec.whatwg.org/ as relatively normative here. What am I missing? -- A*
Re: File API: File's name property
On Sep 3, 2013, at 12:28 PM, Anne van Kesteren wrote: On Tue, Sep 3, 2013 at 5:14 PM, Glenn Maynard gl...@zewt.org wrote: On Tue, Sep 3, 2013 at 10:17 AM, Anne van Kesteren ann...@annevk.nl wrote: I don't think you want those conversion semantics for name. I do think we want the value space for names across different systems to be equivalent, which if we support zip basically means bytes. I don't really understand the suggestion of using a ByteString for File.name. Can you explain how that wouldn't break https://zewt.org/~glenn/picker.html, if the user picks a file named 漢字.txt? ByteString doesn't work. A byte sequence might. If the platform does file names in Unicode it would be converted to bytes using utf-8. Which in fact is how I think we should do File.name. We'll stick to DOMString, but think it should specify a conversion to a byte sequence using utf-8. And, restrict separators such as / and \. -- A*
Re: Extending createObjectUrl to MediaStream?
On Aug 28, 2013, at 9:48 AM, Dominique Hazael-Massieux wrote: Hi WebApps WG, The Media Capture Task Force (responsible for specs that deal with MediaStream objects [1]) has been considering whether one should be able to assign a MediaStream to video and audio elements via an URL obtained via createObjectURL or not, and is seeking feedback on the question. More precisely: A. we define a new srcObject attribute on HTMLMediaElement objects that can take directly a MediaStream object and make it playable video.srcObject = mediastream; B. the spec also supports assigning a MediaStream via the src attribute, via an URL obtained through createObjectURL: video.src = URL.createObjectURL(mediastream); While there are ongoing discussions on how to spec B properly [2] (which will require coordination with the WebApps Working Group), we are first and foremost wondering if that option is needed at all. We are thus looking for input on the use cases for createObjectURL as used for the File API, and whether these use cases would also apply to our MediaStream case. In general, is there a need for any object readable by media elements to support an URL-based approach for consistency with the rest of the platform? The fact that URL.createObjectURL exists at all is hitched to the ongoing use of strings as URLs in places like Image (img src=) and CSS. This also created the discussion about autoRevoke (namely the automatic revocation of these references), for which we added URL.createFor. As it stands, developers should bear the onus of calling URL.revokeObjectURL coupled to URL.createObjectURL, which may be a tall request. MediaStream may not be as ubiquitous as Blob, and you may be able to avoid URL strings altogether. If that's a fair assumption, option A seems like a good plan. If you go the option B route, then you'll have to also think about revocation and URL.createFor, since you'll have the same problem. On balance, I favor Option A, unless you make the case for strings :) If you choose Option B, I'm happy to work with your WG to make sure File API has the right hook for MediaStream objects, per https://www.w3.org/Bugs/Public/show_bug.cgi?id=19594 -- A* -- A*
Re: FileSystem API
On Aug 26, 2013, at 8:01 AM, Arthur Barstow wrote: Hi Eric, Arun, Jonas, All, 2) Is any vendor other than Mozilla actually interested in this proposal? When it was brought up on public-webapps, and at the WebApps F2F, it dropped with a resounding thud. Given the standardization failure of the Chrome FileSystem API, this could be a massive waste of time. Or it could just be a way for Mozilla to document its filesystem API, since we've already got documentation of the Chrome API, but then you don't need to drag public-script-coord into that. These are good questions and points. I don't feel real strongly here re our next steps other than I think we should try to get consensus on a high-level plan to help set expectations accordingly. It seems we have a few options, some are not necessarily mutually exclusive ... 1. Leave Eric's specs in WD state 2. Move Eric's specs to LC-CR (feature set should probably be restricted/limited to what is already implemented in Chrome); block in CR until there are two or more implementations 3. Move Eric's specs to WG Notes and stop work (as was done with Web SQL Database) 4. Merge the two proposals 5. Formally start work on Mozilla proposal knowing there is some overlap with Eric's specs I don't actually think it's possible to merge the two proposals. I'm in favor of option 5. -- A*
Re: File API - Progress - Question about Partial Blob data
On Aug 22, 2013, at 12:07 PM, Jonas Sicking wrote: I think you might have misunderstood my initial comment. I agree that the current partial data solution is not good. I think we should remove it. I'd really like other implementors to weigh in before we remove Partial Blob Data. Cc'ing folks who helped with it. I think we should instead create a better solution in v2 of the API which is based on something other than FileReader and which has the ability to deliver data on the form of here's the data that was loaded since last notification. I agree that we should do a better way. -- A*
Re: File API - Progress - Question about Partial Blob data
On Aug 20, 2013, at 7:13 PM, Aymeric Vitte wrote: The specs says : It can also return partial Blob data. Partial Blob data is the part of the File or Blob that has been read into memory currently; when processing the read method readAsText, partial Blob data is a DOMString that is incremented as more bytes are loaded (a portion of the total) [ProgressEvents], and when processing readAsArrayBuffer partial Blob data is an ArrayBuffer [TypedArrays] object consisting of the bytes loaded so far (a portion of the total)[ProgressEvents]. The list below is normative for the result attribute and is the conformance criteria for this attribute What is the rationale for that? The result attribute should better contain for progress events the latest data read and not the data read from the begining that you could easily reconstitute while the contrary requires more work. Use case: calculate the hash of a file while you are reading it. The ask for deltas to be sent with progress notifications came up on this listserv before -- see the thread starting at http://lists.w3.org/Archives/Public/public-webapps/2013JanMar/0069.html I agree that the deltas model is also useful, but you can see there's some implementation history with XHR here as well. The use case is to receive the file bits as they are constituted from the read (just as with an HTTP request, where you get the bits so far till the rest are constituted). A good way to solve the use case of meaningful deltas might be with the Streams API, still TBD. PS: I did not test it in all browsers, but unless I am using it wrongly, the result attribute is always null for progress events. But not null for the onload ? In many cases, a progress might not fire, depending on file size. -- A*
Re: File* Specifications | Status and some Promises
On Aug 20, 2013, at 8:58 AM, Anne van Kesteren wrote: On Thu, Aug 15, 2013 at 6:49 PM, Arun Ranganathan a...@mozilla.com wrote: ii. A new static method on URL called URL.createAndRevoke has been added that gets defaults right, and does away with autoRevoke, which nobody implemented for URL.createObjectURL. My problem with these names is that they suggest you get a URL object whereas you actually get a string. They also seem longish. I suggest we name the revoking variant just create or a synonym that has less of a suggestion of creating a URL object. I agree, and got the same feedback in off-list correspondence. The ship has already sailed on URL.createObjectURL unfortunately, but we can get the newer function's name right. Jonas proposes URL.createFor in lieu of URL.createAndRevoke. -- A* -- http://annevankesteren.nl/
FileSystem API
Greetings Eric and WG, The Chair and I were discussing setting up repositories for the specifications discussed here (http://lists.w3.org/Archives/Public/public-webapps/2013JulSep/0307.html), notably the FileSystem API and File API v2. Before creating a repository to edit the FileSystem API, we thought we'd check with you about the first proposal, which Chrome implements, and get the Google perspective. You've edited the first FileSystem API proposal, which currently lives here (http://www.w3.org/TR/file-system-api/). Can I create a repository and edit the other proposal for FileSystem API, which currently exists as an email thread (http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html) ? Just checking to see if there are any objections or concerns that would stop a draft or future WG activity. Of course, technical nits should be heard as well, and can proceed concurrently with a draft :) -- A*
File* Specifications | Status and some Promises
Greetings WG, There are three moving proposals in the File* arena, and I thought I'd catpure what these are and what the status of these are. 1. The File API, currently updated as editor's draft at http://dev.w3.org/2006/webapi/FileAPI/ Review strongly encouraged :) Notable fixes are: i. A File constructor has been added, which has been a longstanding request. ii. A new static method on URL called URL.createAndRevoke has been added that gets defaults right, and does away with autoRevoke, which nobody implemented for URL.createObjectURL. iii. A Blob URL Store and a Revocation List, useful for URL Fetch (see https://www.w3.org/Bugs/Public/show_bug.cgi?id=17765). This draft should be ready for LC and should be considered the File API v1. 2. Re-writing File APi to be Promises based, which should be considered a v2. The biggest departure from the current state of technology here would really be that we can have Promises right off the Blob itself, which is zeitgeist thinking about these things (and admittedly makes life easier than an XHR-inspired FileReader, which will be made legacy). The best stab at a strawperson was this one by Jonas: http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0727.html But Stream itself temporarily lives here without too much implementation backbone: https://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm I think we *should* have a Promises-based File API off of Blob, but I'm not totally sure about the Stream API as it is currently written; I'm amenable to editing the File API v2 portion, modulo our proposal for Stream. 3. Then there's the FileSystem API, currently discussed here: http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html This was the upshot of discussions on this listserv, as well as internally at Mozilla. I'm amenable to editing this, which is separate but related to 1. and 2. -- A*
Re: File constructor
On Jul 16, 2013, at 8:40 AM, Anne van Kesteren wrote: On Mon, Jul 15, 2013 at 11:37 PM, Arun Ranganathan a...@mozilla.com wrote: On Jul 15, 2013, at 8:01 PM, Anne van Kesteren wrote: So exposing FormData's contents came up again. Can you send me a pointer to that discussion, so I can plug into it? My bad: https://www.w3.org/Bugs/Public/show_bug.cgi?id=22680 https://www.w3.org/Bugs/Public/show_bug.cgi?id=20887 Sounds like mapping FormData to a File hasn't come up yet. That's the third thread referenced in that bug. Seems like there's not much actual objections. I thought implementers had concerns. I don't think there's actual technical objections (though at one point, I did argue for leaving File as something that held real file objects from the underlying file system, and exposing a name on Blob). But I think a File constructor can be added. -- A*
Re: File constructor
On Jul 15, 2013, at 8:01 PM, Anne van Kesteren wrote: So exposing FormData's contents came up again. Can you send me a pointer to that discussion, so I can plug into it? The natural model is to have FormData be map of string to either a string or File object. This is very much equivalent to exposing a constructor for File to create a Blob with a filename. I was told people objected to that in the past. What were the objections and are they still relevant? Here's a bug with references to the objections( which were really wondering what the right use case is): https://www.w3.org/Bugs/Public/show_bug.cgi?id=20887 Sounds like mapping FormData to a File hasn't come up yet. -- A*
Re: Blob URLs | autoRevoke, defaults, and resolutions
On May 7, 2013, at 10:45 AM, Anne van Kesteren wrote: On Mon, May 6, 2013 at 11:11 PM, Jonas Sicking jo...@sicking.cc wrote: The only thing that's different about XHR is that the first step in my list lives in one function, and the other steps live in another function. Doesn't seem to have any effect on the discussions here other than that we'd need to define which of the two functions does the step which grabs a reference to the Blob. Fair enough. So I guess we can indeed fix this by changing http://fetch.spec.whatwg.org/#concept-fetch to get a reference to the Blob/MediaStream/... before returning early as Arun suggested. \o/ :-) Filed https://www.w3.org/Bugs/Public/show_bug.cgi?id=21955 -- A*
Blob URLs | autoRevoke, defaults, and resolutions
At the recent TPAC for Working Groups held in San Jose, Adrian Bateman, Jonas Sicking and I spent some time taking a look at how to remedy what the spec. says today about Blob URLs, both from the perspective of default behavior and in terms of what correct autoRevoke behavior should be. This email is to summarize those discussions. Blob URLs are used in different parts of the platform today, and are expected to work on the platform wherever URLs do. This includes CSS, MediaStream and MediaSource use cases [1], along with use of 'src='. (Separate discussions about a v2 of the File API spec, including use of a Futures-based model in lieu of the event model, took place, but submitting a LCWD with major interoperability amongst all browsers is a good goal for this draft.) Here's a summary of the Blob URL issues: 1. There's the relatively easy question of defaults. While the spec says that URL.createObjectURL should create a Blob URL which has autoRevoke: true by default [2], there isn't any implementation that supports this, whether that's IE's oneTimeOnly behavior (which is related but different), or Firefox's autoRevoke implementation. Chrome doesn't touch this yet :) The spec. will roll back the default from true to false. At least this matches what implementations do; there's been resistance to changing the default due to shipping applications relying on autoRevoke being false by default, or at least implementor reluctance [1]. Switching the default to false would enable IE, Chrome, andFirefox to have interoperability with URL.createObjectURL(blobArg), though such a default places burdens on web developers to couple create* calls with revoke* calls to not leak Blobs. Jonas proposes a separate method, URL.createAutoRevokeObjectURL, which creates an autoRevoke URL. I'm lukewarm on that :-\ 2. Regardless of the default, there's the hard question of what to do with Blob URL revocation. Glenn / zewt points out that this applies, though perhaps less dramatically, to *manually* revoked Blob URLs, and provides some test cases [3]. Options are: 2a. To meticulously special-case Blob URLs, per Bug 17765 [4]. This calls for a synchronous step attached to wherever URLs are used to peg Blob URL data at fetch, so that the chance of a concurrent revocation doesn't cause things to behave unpredictably. Firefox does a variation of this with keeping channels open, but solving this bug interoperably is going to be very hard, and has to be done in different places across the platform. And even within CSS. This is hard to move forward with. 2b.To adopt an 80-20 rule, and only specify what happens for some cases that seem common, but expressly disallow other cases. This might be a more muted version of Bug 17765, especially if it can't be done within fetch [5]. This could mean that the blob clause for basic fetch[5] only defines some cases where a synchronous fetch can be run (TBD) but expressly disallows others where synchronous fetching is not feasible. This would limit the use of Blob URLs pretty drastically, but might be the only solution. For instance, asynchronous calls accompanying embed, defer etc. might have to be expressly disallowed. It would be great if we do this in fetch [5] :-) Essentially, this might be to do what Firefox does but document what dereference means [6], and be clear about what might break. Most implementors acknowledge that use of Blob URLs simply won't work in some cases (e.g. CSS cases, etc.). We should formalize that; it would involve listing what works explicitly. Anne? 2c. Re-use oneTimeOnly as in IE's behavior for autoRevoke (but call it autoRevoke). But we jettisoned this for race conditions e.g. // This is in IE only img2.src = URL.createObjectURL(fileBlob, {oneTimeOnly: true}); // race now! then fail in IE only img1.src = img2.src; will fail in IE with oneTimeOnly. It appears to fail reliably, but again, dereference URL may not be interoperable here. This is probably not what we should do, but it was worth listing, since it carries the brute force of a shipping implementation, and shows how some % of the market has actively solved this problem :) 3. We can lift origin restrictions in v2 on Blob URL; currently, one shipping implementation (IE) actively relies on origin restrictions, but expressed willingness to phase this out. Most use cases needing Blob data across origins can be met without needing Blob URLs to not be origin restricted. Blob URLs must be unguessable for that to happen, and today, they aren't unguessable in some implementations. -- A* [1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=19594 [2] http://dev.w3.org/2006/webapi/FileAPI/#creating-revoking [3] http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0294.html [4] https://www.w3.org/Bugs/Public/show_bug.cgi?id=17765 [5] http://fetch.spec.whatwg.org/#concept-fetch [6]
Re: File API: Blob.type
On Apr 5, 2013, at 3:17 PM, Alexey Proskuryakov wrote: 03 апр. 2013 г., в 13:11, Arun Ranganathan a...@mozilla.com написал(а): My only concern is that blob.type should never contain parameters. Comparing it to text/plain or image/jpeg should work, and not mysteriously fail a year later when somebody eventually throws a MIME type parameter into the mix. Today, all browsers expose text files at text/plain. If a browser a year from now decides to call text files with a UTF-8 BOM text/plain; charset=UTF-8, it'll break interop. What specifies how a File gets its type? The only requirement I can find is that User agents must not attempt heuristic determination of type, which I think implies that something like inputElement.files[0].type is always for a file chosen by a user via input type=file. The spec. now overreaches a bit :-( Not allowing heuristic mechanisms was merely to restrict encoding determination as per at lease one implementation's experience with it being substandard: https://bugzilla.mozilla.org/show_bug.cgi?id=848842 But now maybe we're going a bit far. Should we standardize how UAs do auto-detect of file type, including something about extensions and some BOM methods? This seems to be complicated and may be unnecessary -- most UAs do this just about right in the absence of a standard. Guessing MIME type from file name or metadata is always a heuristic, as not all platforms will know that archive.sit means application/x-stuffit. At the same time, browsers do autodetect types for many files. We'll need to autodetect when serializing a form for submission anyway, so exposing this information a little earlier only makes sense. I think that these concerns can be resolved by specifying what File.type is more explicitly. The spec can just say that parameters are not allowed in the browser chosen type. That seems sensible! By *not* allowing charset parameters in types determined by UAs, these are now set by web applications only, which may mitigate Glenn's concerns. Maybe the way forward is to leave this to UAs, and: 1. Say UAs should return file type, if known. 2. UAs must not use heuristics or statistical methods to determine encoding and 3. UAs must not set the charset parameter in the returned type for text/plain. This will then defer to the encoding spec. and attempt fallback decoding. Where a web application sets a charset parameter, this will do the right thing for readAsText with fallback decoding. Additionally, determining a blob's file type seems like the most obvious use of this property, and making people say if(blob.type.split(;)[0] == 'text/plain') is simply not a good interface. OK -- you're strongly opinionated on the matter of NOT allowing a charset parameter. I'd like to see if implementers who had an opinion on its usefulness can weigh in -- Darin? Alexey? I do not have a very strong opinion. I like the simpler API of passing parameters through the type attribute, as it's specified currently. This also matches XMLHttpRequest API better. And of course, keeping existing behavior means that we won't break the web. I like it too. We keep charset, but don't let user agents set it for auto-detected files; it can only be set with a Blob constructor or a slice call. Blob.type is a string that can be set by developers and has normative requirements that are not strict tokenization requirements, so I think we're fine here. -- A*
Re: File API: Blob.type
On Mar 19, 2013, at 8:52 PM, Glenn Maynard wrote: On Tue, Mar 19, 2013 at 1:41 PM, Arun Ranganathan a...@mozilla.com wrote: 2.Convert every character in relativeContentType to lower case. I recommend referencing Converting a string to ASCII lowercase in HTML. http://www.whatwg.org/specs/web-apps/current-work/#converted-to-ascii-lowercase Done. 1.If relativeContentType contains any non-ASCII characters, then set relativeContentType to the empty string and return from these substeps. 3.If relativeContentType contains any line break characters like CR or LF or any CTLs or separators, then set relativeContentType to the empty string and return from these substeps. #3 is too vague. I recommend combining #1 and #3, saying: If any character in relativeContentType outside of the range U+0020 to U+007E. That's the printable ASCII range, and excludes all control characters. Done (+ thanks). 4.Parse relativeContentType as an RFC2616 media-type, tokenizing it according to the ABNF for media-type [RFC2616] with the ASCII / character separating tokens representing the type and subtype productions. If relativeContentType cannot be tokenized according to the ABNF for media-type [RFC2616], then set relativeContentType to the empty string and return from these substeps. I'm not sure we should be this strict. I'd lean towards keeping it simple, allowing any string at all as long as it contains only lowercase, printable ASCII. Done -- we restrict it now, but don't mandate tokenization along the lines of RFC2616. You don't need to say The following requirements are normative for this parameter. That's what the normative language that follows (must) means. Done. My only concern is that blob.type should never contain parameters. Comparing it to text/plain or image/jpeg should work, and not mysteriously fail a year later when somebody eventually throws a MIME type parameter into the mix. Today, all browsers expose text files at text/plain. If a browser a year from now decides to call text files with a UTF-8 BOM text/plain; charset=UTF-8, it'll break interop. Additionally, determining a blob's file type seems like the most obvious use of this property, and making people say if(blob.type.split(;)[0] == 'text/plain') is simply not a good interface. OK -- you're strongly opinionated on the matter of NOT allowing a charset parameter. I'd like to see if implementers who had an opinion on its usefulness can weigh in -- Darin? Alexey? http://dev.w3.org/2006/webapi/FileAPI/ -- A*
Re: File API: why is there same-origin restriction on blob URLs?
On Mar 28, 2013, at 1:55 AM, Anne van Kesteren wrote: On Wed, Mar 27, 2013 at 8:16 PM, Arun Ranganathan a...@mozilla.com wrote: They're very different than data URLs. What's a good use case for making them cross-origin, that isn't addressed by use of postMessage? Well, the question we started with was why they're restricted to same origin. That does not appear to be answered yet. The restriction was due to the (perhaps misguided?) safety assumption that it was prudent to restrict file references via this scheme to the origin invoking URL.createObjectURL. The initial proposal was scoped to origin, and dates to some antiquity in email from Hixie, which said: URL in this scheme would have an intrinsic origin that is equal to the origin of the script context (the first script in HTML5 terms) that invoked the API to get the URL. This URL could then be passed around as a string and would be treated as a resource from the same origin as that script, and could thus be used with img and canvas and so on. http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/1110.html We can relax this requirement, since it has been pointed out that it is a bit restrictive, but I'd like to ensure that Bad Things won't happen. Implementor feedback is welcome. As for a use case, you could have a widget that does something given a URL. E.g. put Googly eyes on the image resource. This widget could be embedded anywhere and all you'd need to do is postMessage() it a URL. Requiring a special code path for URLs generated from a Blob seems annoying for both the consumer of the widget and the producer of it. Glenn adds that there may need to be some special casing anyway (http://lists.w3.org/Archives/Public/public-webapps/2013JanMar/1038.html), with respect to revocation, but I'm open to relaxing the origin restriction if assumptions about this being prudent prove to be invalid. -- A*
Re: File API: why is there same-origin restriction on blob URLs?
On Mar 26, 2013, at 8:30 PM, Jonas Sicking wrote: On Tue, Mar 26, 2013 at 2:17 PM, Anne van Kesteren ann...@annevk.nl wrote: Hi, Is there any particular reason why we restrict blob URLs to the same origin as the script that created them? In effect they are pretty much like capability URLs (containing an unguessable token). So if someone decides to share one, that should be okay I think. This would be useful in the context of sandboxed code (iframe sandbox) and presumably elsewhere too. I think the original concern was that implementations might not be able to reliably generate unguessable URLs. Potentially that's something that we could require though. We already require this -- opaque strings should be globally unique. However we'd still need to nail down what the new behavior should be. Should it behave like data: URLs? The main advantage of those is that implementations still don't agree on how those should behave. They're very different than data URLs. What's a good use case for making them cross-origin, that isn't addressed by use of postMessage? -- A*
Re: File API for Review
On Mar 25, 2013, at 1:52 PM, Anne van Kesteren wrote: On Mon, Mar 25, 2013 at 5:45 PM, Arun Ranganathan aranganat...@mozilla.com wrote: So just to be clear, do you think we should remove 500-style responses altogether, and *completely defer* to network error, which essentially involves throwing on expired / revoked / invalid Blob URLs? I'm ok with that if so. Yeah, I would prefer if we used network error consistently throughout the platform for something went wrong with this URL rather than try to re-imagine a non-HTTP situation in terms of HTTP. Because it gives nicer to work with events and such out of the box and because it makes exposing more detailed information in the future easier. Done. http://dev.w3.org/2006/webapi/FileAPI/#NetworkError -- A*
Re: File API for Review
- Anne vK said: - On Tue, Mar 19, 2013 at 5:57 PM, Arun Ranganathan a...@mozilla.com wrote: On Feb 13, 2013, at 11:37 AM, Anne van Kesteren wrote: We're not actually leaving what exactly to return open to implementors. They *must* return a 500. They *may* additionally provide a message, which is akin to console messages. No, an HTTP status message is an actual thing and exposed via XMLHttpRequest. You're right; this should be defined better in the File API. Also, networkerror isn't really strongly defined; XMLHttpRequest uses this to throw on network errors (NetworkError), and there isn't currently a Fetch in HTML that leverages networkerrors. This is not obviously reusable here in the blob: URL context. I think it is. Network error is for whenever something goes wrong with the request. Otherwise this would give a load event rather than an error event. data: URLs use network errors too now: http://xhr.spec.whatwg.org/#data:-urls-and-http So just to be clear, do you think we should remove 500-style responses altogether, and *completely defer* to network error, which essentially involves throwing on expired / revoked / invalid Blob URLs? I'm ok with that if so. -- A*
Re: File API for Review
On Feb 13, 2013, at 11:37 AM, Anne van Kesteren wrote: Specification bugs still exist I think we should rename URI to URL. That's what everyone is converging on. Done. I'm also not convinced that leaving what exactly to return in the HTTP scenario open to implementors is a good thing. (There isn't an HTTP scenario here). So what I actually think we should do here is treat this as a networkerror. XMLHttpRequest already knows about that concept and every other end point also deals with network errors in a predictable and standardized way. Phrasing such as Act as if a network error occurred seems sufficient for now (until Fetch provides hooks). We're not actually leaving what exactly to return open to implementors. They *must* return a 500. They *may* additionally provide a message, which is akin to console messages. Also, networkerror isn't really strongly defined; XMLHttpRequest uses this to throw on network errors (NetworkError), and there isn't currently a Fetch in HTML that leverages networkerrors. This is not obviously reusable here in the blob: URL context. The spec. doesn't need to mention anything about throwing -- we're returning a 500 if a blob: URL cannot be dereferenced, not throwing an exception. This is specific, and not left to implementations to determine. Just like HTML, CSS, etc. this specification should defer to http://encoding.spec.whatwg.org/ for its encoding related requirements. Done (and uses most of your proposed changes): http://dev.w3.org/2006/webapi/FileAPI/#encoding-determination I don't think we should throw for limitations on URL length. We always leave undefined lengths unaddressed in specifications, including with regards to how to handle them. Done. -- A*
Re: File API: Blob.type
On Mar 7, 2013, at 7:19 PM, Glenn Maynard wrote: Chrome, at least, throws on new Blob([], {type: 漢字}), as well as lowercasing the string. Stricter rules are in place for type both while constructing Blob and for slice calls: http://dev.w3.org/2006/webapi/FileAPI/#constructorBlob and http://dev.w3.org/2006/webapi/FileAPI/#slide-method-algo I agree with previous comments you've made about ByteString not solving any problems that Anne vK brings up; instead, I think using DOMString is probably ok, with tighter rules on what is valid and what should be ignored. Throwing a SyntaxError might be overkill to developers and a bit too punitive; instead, I advocate sticking with the original spirit of the opaque string idea and ignoring bad use of type. A couple points: - I disagree that we should discourage comparing against Blob.type, but ultimately it's such an obvious use of the property, people will do it whether it's encouraged or not. I'd never give it a second thought, since that appears to be its very purpose. Web APIs should be designed defensively around how people will actually use the API, not how we wish they would. Unless lots of Blob.type parameters actually include parameters, code will break unexpectedly when it ends up encountering one. - The RFC defines a protocol (Content-Type), not a JavaScript API, and a good protocols are rarely good APIs. Having Blob.type be the literal value of a Content-Type header isn't an elegant API. You shouldn't need to do parsing of a string value to extract text/plain, and you shouldn't have to do serialization to get text/plain; charset=UTF-8. So the type attribute of a Blob object isn't the *literal* value of the header; it's the type of the Blob, expressed as a MIME type. When dereferencing Blob URLs, you get this type back with the Content-Type header, as you do normally in HTTP scenarios. This is a well-understood behavior, and I agree with points you've made about not being beholden to the RFC when designing an API. I think the question here is whether or not to include *separate attributes* on the Blob interface for the rarely used Charset Parameter, namely anything after the semicolon in MIME types of the sort: text/plain;charset=UTF-8. I've considered all your arguments by way of developer advocacy, and actually think we'll do developers a disservice by adding to the Blob interface: 1. The Charset Parameter consideration applies only to text/plain. There are numerous other MIME types that don't use it: application/*, audio/*, image/*, video/*, etc. Complicating the interface on the off-chance that a stray use of the Charset parameter breaks a direct equality comparison is too much API for too little. 2. The Charset Parameter even in the context of text/plain isn't common enough to warrant a special case for text/plain within the API. 3. In general, it's a pretty stable assumption to conclude that developers will expect type to be surfaced later along with Content-Type when dereferencing a Blob URI. I don't think we've made an assumption that's terribly galling. -- A*
Re: File API: Blob.type
Alexey, On Mar 7, 2013, at 3:02 PM, Alexey Proskuryakov wrote: The current File API spec seems to have a mismatch between type in BlobPropertyBag, and type as Blob attribute. The latter declaratively states that the type is an ASCII lower case string. As mentioned by Glenn before, WebKit interpreted this by raising an exception in constructor for non-ASCII input, and lowercasing the string. I think that this is a reasonable reading of the spec. I'd be fine with raising exceptions for invalid types more eagerly. This is the text in question: (1) type, a DOMString which corresponds to the Blob object's type attribute. If not the empty string, user agents must treat it as an RFC2616 media-type [RFC2616], and as an opaque string that can be ignored if it is an invalid media-type. This value must be used as the Content-Type header when dereferencing a Blob URI. (2) type The ASCII-encoded string in lower case representing the media type of the Blob, expressed as an RFC2046 MIME type [RFC2046]. On getting, conforming user agents must return the MIME type of the Blob, if it is known. If conforming user agents cannot determine the media type of the Blob, they must return the empty string. A string is a valid MIME type if it matches the media-type token defined in section 3.7 Media Types of RFC 2616 [RFC2616]. If not the empty string, user agents must treat it as an RFC2616 media-type [RFC2616], and as an opaque string that can be ignored if it is an invalid media-type. This value must be used as the Content-Type header when dereferencing a Blob URI. This is now clarified; the mismatch is a spec. bug. Thanks for pointing this out. It would be helpful to have the terminology corrected, and to have this generally clarified - for example, validity is mentioned here, but seems to be unused. Conditions for validity have been clarified; this doesn't warrant throwing a SyntaxError, but it does specify when implementations should ignore poor use of MIME type strings, e.g. here's additional clarification in the slice call: http://dev.w3.org/2006/webapi/FileAPI/#slide-method-algo It seems pretty clear from normative text that charset parameter is supposed to work. A non-normative example supports that too. I agree with Arun that this seems best to keep as is. +1. However, https://bugs.webkit.org/show_bug.cgi?id=111380 is about a different case - it's about posting multipart form data that has Blob elements with invalid media-types. I'm not even sure which spec is in charge of this behavior - I don't think that anything anywhere says that Blob.type affects media-type of posted multipart data, even though that's obviously the intention. XMLHttpRequest spec defers to HTML, which defers to RFC2388, which mentions files returned via filling out a form, but not Blobs (which is no surprise given its age). In fact, I'm not sure if Blob.type should influence the type of multipart form data. Consider the concatenation of several Blobs into a new Blob, as the Blob constructor allows. What should the type of a newly constructed Blob be, if it consists of several differently typed Blobs? The spec. suggests disregarding the type of each Blob, but encourages the right use of type within the Blob constructor. I'm also not sure multipart form data falls under the aegis of the File API, but at least Blobs with invalid types is the same us having no type now (empty string). Making Blobs only hold valid media-types would solve practical issues, but it would be helpful to know what formally defines multipart data serialization with blobs. We also previously had https://bugs.webkit.org/attachment.cgi?id=177736action=review for sending non-multipart data. Back then, we determined that Content-Type: should be sent when the value is invalid. I'm no longer sure if that's right. For this case, XMLHttpRequest authoritatively defines the behavior, although heavily leaning on File API to decide when the type attribute is empty: If the object's type attribute is not the empty string let mime type be its value. Note that mime type is then directly used as default media-type for Content-Type header, but it's not parsed to set encoding variable. The encoding could be needed to update a charset in author provided Content-Type header field in later steps of the algorithm. This is probably not right, as Blob should know its encoding better than code that sets header fields on an XMLHttpRequest object. Yes, but implementations can't heuristically determine a Blob's type now. Type has to be specified correctly or ignored. What Blob should know is now as good as what it is constructed to have as its type, though at read time, thanks to the Encoding Spec, we can determine a fallback encoding. -- A*
Re: File API for Review
On Mar 8, 2013, at 8:28 AM, Henri Sivonen wrote: Additionally, I think http://www.w3.org/TR/FileAPI/#dfn-type should clarify that the browser must not use statistical methods to guess the charset parameter part of the type as part of determining the type. Firefox currently asks magic 8-ball, but the magic 8-ball is broken. AFAICT, WebKit does not guess, so I hope it's possible to remove the guessing from Firefox. (The guessing in Firefox relies on a big chunk of legacy code that's broken and shows no signs of ever getting fixed properly. The File API is currently the only thing in Firefox that exposes the mysterious behavior of said legacy code to the Web using the default settings of Firefox, so I'm hoping to remove the big chunk of legacy code instead of fixing it properly.) I think we can jettison Fx's use of magic 8-ball code with this update: http://dev.w3.org/2006/webapi/FileAPI/#dfn-type Additional spec'ing on when to ignore the string if provided incorrectly will follow. -- A*
Re: File API: Blob.type
On Mar 6, 2013, at 7:42 PM, Glenn Maynard wrote: On Wed, Mar 6, 2013 at 8:29 AM, Anne van Kesteren ann...@annevk.nl wrote: On Wed, Mar 6, 2013 at 2:21 PM, Glenn Maynard gl...@zewt.org wrote: Blob.type is a MIME type, not a Content-Type header. It's a string of codepoints, not a series of bytes. XHR is a protocol-level API, so maybe it makes sense there, but it doesn't make sense for Blob. It's a Content-Type header value and should have those restrictions. It's not a Content-Type header, it's a MIME type. That's part of a Content-Type header, but they're not the same thing. In fact, the intent is that the value of Blob.type is reflected in the Content-Type, and that setting Blob.type means that when fetching that Blob as a blob: you'll get the value of Blob.type in the Content-Type header. This model *did* allow for charset params -- it always has (perhaps not advertised, but it always has). At some point there was a draft that specified *strict* parsing for compliance with RFC2046, including tokenization (/) and eliminating non-ASCII cruft. But we scrapped that because bugs in all major browser projects showed that this spec. text was callously ignored. And I didn't want to spec. fiction, so we went with the current model for Blob.type, which is, as Anne points out, pretty lax. That doesn't make sense. Blob.type isn't a string of bytes, it's a string of Unicode codepoints that happens to be restricted to the ASCII range. Applying WebKit's validity checks (with the addition of disallowing nonprintable characters) will make it have the restrictions you want; ByteString has nothing to do with this. I'm in favor of introducing stricter rules for Blob.type, and I'm also in favor of allowing charset params; Glenn's example of 'if(blob.type == text/plain)' will break, but I don't think we should be encouraging strict equality comparisons on blob.type (and in fact, should *discourage* it as a practice). But I'm not sure about why we'd choose ByteString in lieu of being strict with what characters are allowed within DOMString. Anne, can you shed some light on this? And of course we should eliminate CR + LF as a possibility at constructor invocation time, possibly by throwing. Glenn: I think that introducing a separate interface for other parameters actually takes away from the elegance of a simple Blob.type. The RFC doesn't separate them, and I'm not sure we should either. My reading of the RFC is that parameters *are an intrinsic part of* the MIME type. -- A* - Original Message - On Wed, Mar 6, 2013 at 8:29 AM, Anne van Kesteren ann...@annevk.nl wrote: On Wed, Mar 6, 2013 at 2:21 PM, Glenn Maynard gl...@zewt.org wrote: Blob.type is a MIME type, not a Content-Type header. It's a string of codepoints, not a series of bytes. XHR is a protocol-level API, so maybe it makes sense there, but it doesn't make sense for Blob. It's a Content-Type header value and should have those restrictions. It's not a Content-Type header, it's a MIME type. That's part of a Content-Type header, but they're not the same thing. But String vs. ByteString has nothing to do with the restrictions applied to it. Making it a ByteString plus additional restrictions will make it do as required. That doesn't make sense. Blob.type isn't a string of bytes, it's a string of Unicode codepoints that happens to be restricted to the ASCII range. Applying WebKit's validity checks (with the addition of disallowing nonprintable characters) will make it have the restrictions you want; ByteString has nothing to do with this. On Wed, Mar 6, 2013 at 11:47 AM, Darin Fisher da...@chromium.org wrote: So the intent is to allow specifying attributes like charset? That sounds useful. I don't think so. This isn't very well-defined by RFC2046 (it seems vague about the relationship of parameters to MIME types), but I'm pretty sure Blob.type is meant to be only a MIME type, not a MIME type plus content-type parameters. Also, it would lead to a poor API: you could no longer simply say 'if(blob.type == text/plain)'; you'd have to parse it out yourself (which I expect nobody is actually doing). Other parameters should have a separate interface, eg. blob.typeParameters.charset = UTF-8, if we want that. -- Glenn Maynard
Re: File API for Review
Anne, On Wed, Feb 6, 2013 at 7:58 PM, Arun Ranganathan a...@mozilla.com wrote: 3. Progress events have been clarified. You're still using the old IDL syntax for event handlers. Fixed. I think we should rename URI to URL. That's what everyone is converging on. Fixed. I'm also not convinced that leaving what exactly to return in the HTTP scenario open to implementors is a good thing. We've been through such things before and learned that handwaving is bad. Lets just pick something. Just to be clear, are you referring to the 500 Error Condition for Blob URLs? If so, the only handwaving is about the text of the error message. I'm happy to tighten even this. Just like HTML, CSS, etc. this specification should defer to http://encoding.spec.whatwg.org/ for its encoding related requirements. I fully agree that what we've currently got, which favors a heuristic guessing model for encoding, and forces UTF-8 in a void, isn't sufficient. AND I agree that the encoding spec. is much more detailed. But what exactly does deferring to it entail? Right now, the specification encourages user agents to get encoding from: 1. The encoding parameter supplied with the readAsText. 2. A byte order detection heuristic, if 1. is missing. 3. The charset component of Blob.type, if provided and if 1. and 2. yield no result. 4. Just use utf-8 if 1, 2, and 3 yield no result. Under the encoding spec., it returns failure if encoding isn't valid, and it returns failure if the BOM check fails. So should the spec. say something about throwing? I don't think we should throw for limitations on URL length. We always leave undefined lengths unaddressed in specifications, including with regards to how to handle them. OK. -- A*
Re: Please add constructors for File and FileList
Greetings Victor! On Dec 10, 2012, at 12:02 PM, Victor Costan wrote: Dear Web Applications WG, 1) Please add a File constructor. This has cropped up a few times :) I've logged a spec bug for this feature: https://www.w3.org/Bugs/Public/show_bug.cgi?id=20887 Could you flesh out your original use case a bit more? As currently expressed, it sums up to I could write better unit tests which doesn't constitute what I'm looking for in a use case. It strikes me that the chief ask is to bind a Blob to name. This would make life simpler with FormData and with existing server applications. It's been pointed out that the barrier between Blob and File is pretty thin, and I'm open to jettisoning the thought pattern that we should think of File objects as strictly filesystem backed (on disk). So, what if we allowed the Blob constructor to take a name also? This might allow Blobs to fall into the 80 of the 80-20 rule :) Could you update the bug (or this listserv) with better use cases? I'm a bit less upbeat about: 2) Please add a FileList constructor. What I really want is some way to add files to an input type=file listed in http://wiki.whatwg.org/wiki/New_Features_Awaiting_Implementation_Interest I think that one reasonable way to get there is to have a FileList constructor that takes an array of File instances, and have the ability to assign that FileList to the files attribute of the input. This avoids making FileList mutable. This would also help me write better tests for my code. Currently, input type=file is the only form field whose value can't be set inside JavaScript, so I can't write automated tests for input type=file-related code. Asides from improving testing, this would allow me to implement the following _easily_: * filters for uploaded content (e.g. resize a big image before uploading) * saving the file a user selected in an IndexedDB, and loading it back into the input type=file if the page is accidentally refreshed These features can be implemented without FileList support, but they impact the application's design. For example, filters can be implemented using FormData from XMLHttpRequest level 2 to upload Blobs, but switching from plain form submission to XHR-based submission can impact other parts of the application code, whereas changing the files behind the input would be a localized change. As it stands currently, a FileList object typically stems from user action. For instance, interacting with the file picker or with drag and drop. In general, this model is desirable for security reasons -- the legacy HTML form and file picker ensures that web applications aren't doing things they shouldn't be doing with the underlying file system without user participation. I don't actually think it is desirable to modify input type = file… from within JavaScript. I'd like a better understanding of the broader implications of a FileList constructor, just out of abundant caution. I have similar concerns about https://www.w3.org/Bugs/Public/show_bug.cgi?id=17125 which is also filed on the FileList object, but these seem easier to manage. Hope to hear back, -- A*
File API for Review
Greetings WebApps WG! Review on the File API is encouraged: http://dev.w3.org/2006/webapi/FileAPI/ A few substantial changes that might need particular attention before we initiate a call for LCWD or something comparably official: 1. autoRevoke behavior of Blob URIs has changed. Previous drafts made the autoRevoke behavior on by default (by consensus), but didn't harness revocation to a suitable and unambiguous scope. Thanks to a fix to HTML[1] we can leverage the global script clean-up jobs list, to which we add revocation of Blob URIs. Blob URIs are thus either scoped to the next time global script clean-up jobs are processed (by default), OR when document unloading steps are processed if the developer opts out of the default but never pairs it with a call to URL.revokeObjectURL, OR when URL.revokeObjectURL is called. In particular, this behavior defers from shipping implementations such as IE10. This is possibly the biggest change: http://dev.w3.org/2006/webapi/FileAPI/#lifeTime http://dev.w3.org/2006/webapi/FileAPI/#creating-revoking 2. An utility to smooth line ending variations (Unix vs. Windows) has been added, but is an orphan interface currently. Nobody's fussed over this, and it might not be a problem at all, but I'd like to draw your attention to it :) While currently only relevant for DOMString arguments to the Blob constructor, we might work with the utility to add other arguments (including ArrayBufferViews, etc.). http://dev.w3.org/2006/webapi/FileAPI/#convenienceAPI 3. Progress events have been clarified. 4. readAsDataURL currently makes progress notifications throughout the read, but only returns a non-null return value at the end of the read. http://dev.w3.org/2006/webapi/FileAPI/#readAsDataURL Specification bugs still exist, and this draft doesn't address all open bugs [2]; further bugs and issues, if any, should be logged or discussed on this listserv. The remaining bugs present use cases that aren't major enough to warrant feature extensions. -- A* [1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=19554 [2] https://www.w3.org/Bugs/Public/buglist.cgi?product=WebAppsWGcomponent=File%20APIresolution=---list_id=4913
Re: [File API] About Partial Blob Data, XHR and Streams API
On Jan 22, 2013, at 9:42 AM, Cyril Concolato wrote: Hi Arun, Le 22/01/2013 15:04, Arun Ranganathan a écrit : Hi Cyril, 1) I'm wondering why in progressive mode, does the spec say: partial Blob data is an ArrayBuffer [TypedArrays] object consisting of the bytes loaded so far. Why isn't it the bytes loaded since the previous progress event? AR: It is always a new ArrayBuffer. The phraseology so far could be replaced by bytes loaded since the previous progress event though I'm not always sure that will be the case. I understood from Jonas' answer that it was a new ArrayBuffer. What remained unclear was the content of the ArrayBuffer: all the data from the beginning of the read operation (which was problematic), or only the data read since the previous progress event (which I prefer). If, as you say, this is latter option, then please fix the spec. as so far is very ambiguous. Hi Cyril: the model you propose, of deltas since the last read, is actually *not* what the spec's intent is. Your model does have advantages, but released software follows the former model, namely *all* the data since the beginning of the read operation. The partial results consist of all the bytes, starting from the beginning, but perhaps not the last byte, depending on progress event and file size. I'll have to clarify the prose, since from your question, it is clear that this isn't well described. -- A*
Re: [File API] About Partial Blob Data, XHR and Streams API
Hi Cyril, 1) I'm wondering why in progressive mode, does the spec say: partial Blob data is an ArrayBuffer [ TypedArrays ] object consisting of the bytes loaded so far . Why isn't it the bytes loaded since the previous progress event? AR: It is always a new ArrayBuffer. The phraseology so far could be replaced by bytes loaded since the previous progress event though I'm not always sure that will be the case. In my use case, the binary data resource might have an infinite size, in which case the result objects will grow forever. I looked at the Streams API [1] to see if there was any difference for that but I couldn't see any. Reading with the FileReader interface a Stream (dynamic length) or a Blob (fixed length) seems to always return the whole content. AR: Here, do you mean, you never get a progressevent other than load and loadend in your tests? Certainly, if you had binary data of infinite size, you'll get several result objects. The file API, particularly FileReader, shouldn't be used in streaming scenarios. -- A*
Re: [admin] Publishing specs before TPAC: CfC start deadline is Oct 15
- Original Message - On 10/9/12 4:13 PM, ext Arun Ranganathan wrote: On Sep 26, 2012, at 10:27 AM, Arthur Barstow wrote: * File API - Arun can you get this spec ready for LC by October 15? Yes. ATM, File API has 8 open bugs [1]. I've rationalized these down to 2 bugs. 1. Bug 17125[1] is a feature that we should mark v.next; it calls for the ability to retroactively unselect an erroneous selection that is present in FileList. I don't think this is a pressing feature. 2. Bug 19554[2] is a spec. request feature in WHATWG/HTML5, especially useful for autoRevoke semantics for Blob URLs. I'm not sure whether autoRevoke is at risk because of this bug, since implementations have shown that stable state (what the spec. uses now) is problematic for autoRevoke. But I'll let this be discussed in LC commentary or in upcoming TPAC discussions. P.S. Dom's WebIDL checker reports a bug in the Blob constructor Thanks for catching that :) I think that particular bug is fixed. And I'm sorry this is a day late (e.g. not ready by Oct. 15). I've gotten it PubReady, using a push out date of Oct. 18. Hope that works: http://dev.w3.org/2006/webapi/FileAPI/ -- A* [1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=17125 [2] https://www.w3.org/Bugs/Public/show_bug.cgi?id=19554
Re: [admin] Publishing specs before TPAC: CfC start deadline is Oct 15
On Sep 26, 2012, at 10:27 AM, Arthur Barstow wrote: * File API - Arun can you get this spec ready for LC by October 15? Yes. -- A*
Re: [FileAPI] Blob constructor should probably take a sequence, not an IDL array object
On Sep 11, 2012, at 1:07 AM, Cameron McCormack wrote: Arun Ranganathan: I've pinged heycam to see if this is a proper use of the sequence type. I'm not sure it allows for such a variation in parameters. I agree with Boris, it makes sense to use sequence here. Whenever you just want to take a list of values in an operation argument, and you don't want to keep a reference to a platform array object, you should use a sequence. Done. http://dev.w3.org/2006/webapi/FileAPI/#dfn-Blob -- A*
Re: [FileAPI] blob: protocol need a content-length header
Benjamin, I filed the following: https://www.w3.org/Bugs/Public/show_bug.cgi?id=18548 I think we should add Content-Length. -- A* On Aug 12, 2012, at 5:41 PM, Benjamin BERNARD wrote: I build de demo script (for firefox) here : http://experiments.benvii.com/blob_content_length/ You will also notice that the player's load event isn't called. Content-Length should be added to firefox (maybe open a ticket on https://bugzilla.mozilla.org/) but it should also be recommended in the section 11.7.3. Request and Response Headers of the spec. Thanks for responding. Benjamin BERNARD Le 12/08/2012 21:23, Jonas Sicking a écrit : On Sun, Aug 12, 2012 at 2:56 AM, Benjamin BERNARD benjamin.bern...@benvii.com wrote: Hi, I was developing an offline music web App when I discover that is no Content-length header specified here : http://www.w3.org/TR/FileAPI/#ProtocolExamples So when you play an audio/video file stored as a blob (under a blob URI) it's considered by the player as streaming content which means you can't get the duration of a song for instance (it has an infinite duration). I think it might be the consequence of not providing a content-length header. I experienced it using Firefox I heard Internet Explorer already provide this header. Moreover, I don't understand why there is no content-length header recommended in the spec because when you use URL.createObjectURL(blob), blob has a finished size (correct me if I'm wrong). So a content-length header should also be provided and recommended in the spec. Yes, I agree, we should have a content-length header similar to the content-type header. In Gecko things are a bit complicated because we don't have headers on anything but http channels. But we do have the concept of a length of a response for all channels so that should take care of it. Not sure off the top of my head why it doesn't. Filing a bug with an example would be great. / Jonas
Re: [File API] File behavior under modification
On Jul 11, 2012, at 10:02 PM, Glenn Maynard wrote: What's the main problem with it being nullable? A fabricated date seems strange, but instead of being nullable we could spec what the fabricated date is. I'm just not totally sure what the pros and cons are here. If you call d.getYear() and d is null, you get an exception, which has a very high chance of breaking the app if it's not checked for. Giving an arbitrary (but well-defined) default is much less likely to break things that badly. This is much more important when the null case is rare, because nobody is going to remember to test for it. Can you log a bug so that I can provide guidance for this in spec? Sorry, but to avoid rereading the thread, a bug for what exactly? Do you need anything beyond https://www.w3.org/Bugs/Public/show_bug.cgi?id=17746? Bug 17746 is for better definitions of snapshots. I minted Bug 17762 (https://www.w3.org/Bugs/Public/show_bug.cgi?id=17762) to replace a nullable Date with something better. Strawperson suggestions welcome for what this arbitrary date should be, or else I may arbitrarily choose an unlikely Towel Day in the past (http://towelday.org/ -- don't panic). -- A*
Re: [File API] File behavior under modification
Glenn: On May 21, 2012, at 9:44 PM, Glenn Maynard wrote: On Mon, May 21, 2012 at 6:05 PM, Eric U er...@google.com wrote: According to the latest editor's draft [1], a File object must always return an accurate lastModifiedDate if at all possible. On getting, if user agents can make this information available, this MUST return a new Date[HTML] object initialized to the last modified date of the file; otherwise, this MUST return null. However, if the underlying file has been modified since the creation of the File, reads processed on the File must throw exceptions or fire error events. ...if the file has been modified on disk since the File object reference is created, user agents MUST throw a NotReadableError... (I wish this spec would remove the screaming MUSTs; HTML doesn't do this anymore, and it's so much easier to read.) WHAT? Heh :) Point well taken. I've muted the musts. These seem somewhat contradictory...you can always look at the modification time and see that it's changed, but if you try to read it after a change, it blows up. The non-normative text about security concerns makes me think that perhaps both types of operations should fail if the file has changed [... guarding against modifications of files on disk after a selection has taken place]. That may not be necessary, but if it's not, I think we should call it out in non-normative text that explains why you can read the mod time and not the data. I think lastModifiedDate should never change. It should be the mtime of the version of the file that the File represents a snapshot of. This avoids synchronicity issues: reading the value twice in a row in the same script should never give a different value or throw the second time but not the first, because that exposes the multithreading nature of the filesystem. It also avoids implying that this attribute needs to perform synchronous I/O in the middle of script execution to check the file's current timestamp, which of course should never happen. If we want to allow querying whether the mtime of a file has changed, it should be done with a new asynchronous API. I'm not sure that's needed, though, since you can always just read one byte from the file; if it fails, the file has changed. I agree that making snapshotting clearer might be a good idea. It is true that reading size and lastModifiedDate are synchronous, but this seemed a small trade-off compared to data reads. My instinct is that an asynchronous API for mtime is overkill. (While I'm thining about it, does lastModifiedDate really need to be nullable? Systems without file timestamps are so rare that it's probably better to require them to fabricate a date, so we don't introduce bugs into people's code for rare cases.) What's the main problem with it being nullable? A fabricated date seems strange, but instead of being nullable we could spec what the fabricated date is. I'm just not totally sure what the pros and cons are here. This came up in https://bugs.webkit.org/show_bug.cgi?id=86811; I believe WebKit is currently noncompliant with this part of the spec, and we were debating the correct behavior. Currently WebKit delays grabbing the modification time of the file until it's been referenced by a read or slice(), so it won't notice modifications that happen between selection and read. This sounds very wrong to me. If I open a File in a page (eg. select it with an input), the model is that I'm giving it access to the file as it was at the time I dragged it in. If the snapshot is delayed until the first read, the page will be able to see changes made later, as long as it doesn't touch the file immediately. That breaks the whole security model. Strong +1. Also, slice() is a synchronous API, so it should never cause blocking file I/O. That's fundamental to the API. That was done because the slice creates a File object reference, but in my reading creating the File referring to the file should be the time of the snapshot, not creating a Blob referring to a File. FWIW, agreed. Can you log a bug so that I can provide guidance for this in spec? -- A*
Re: [File API] File behavior under modification
Glenn, On May 22, 2012, at 11:48 AM, Glenn Maynard wrote: On Tue, May 22, 2012 at 1:41 AM, Kinuko Yasuda kin...@google.com wrote: In my understanding WebKit's behavior is querying the metadata / reading the content as lazy as possible, partly because the spec was/is ambiguous (especially about when the file metadata should be captured) and also we didn't want to break existing Web applications. But the synchronicity of the current .size, .modificationTime and .slice() implementation has been a big headache and I'm all happy if we can make the behavior clearer. The whole point of the Blob API is to never, ever do file I/O synchronously in the UI thread. (I think the read into memory at the time a read operation is initiated bit is what's confusing things. I don't think it was intended to mean take the snapshot at the first read operation, but that's what it sounds like.) Some other issues: snip / - The access restrictions applied to File also need to be clearly applied to Blobs created with slice(). - modified on disk since the File object reference is created seems incorrect. If you structured clone a File (eg. postMessage), you create a new File object. That shouldn't refresh the snapshot; you should still only be able to access the file as it was when you received the *original* File. I'd suggest defining this more precisely, and at the level of Blob. Here's an attempt: - In section 6, add: Each Blob has a snapshot state, which is initially set to the state of the underlying storage. Note: this value is preserved through structured clone. - Also in section 6, add: If, at the time of any read operation on the Blob, the state of the underlying storage containing the Blob is not equal to snapshot state, the read must fail with a NotReadableError. - In 6.3.2. The slice method, add: Let the snapshot state of A be the snapshot state of O. (This means the snapshot state follows through slices.) - In 7.1, remove For synchronous reads The requirement to fail is now defined by section 6, and the way it fails (passing on the NotReadableError or an error event) should be defined by the associated read operation. This stores a conceptual snapshot of the underlying storage at the time the Blob (or File) is created. The snapshot state is purely conceptual, representing the logical underlying snapshot that the Blob represents. If it represents a file on disk, and the file changes (or is deleted), it has a new snapshot state. This approach puts the requirement at the Blob level. For Blobs whose underlying storage can never be modified (regular, old-fashioned blobs), it has no effect, since the snapshot state never changes. The snapshot is made when the a Blob (or File) is constructed, except for slice() and structured clone, which copy the source object's state. These are really solid proposals, and I've filed Bug 17746 to keep track of this. https://www.w3.org/Bugs/Public/show_bug.cgi?id=17746 -- A* -- Glenn Maynard
Re: [File API] File behavior under modification
On May 23, 2012, at 9:58 AM, Glenn Maynard wrote: On Wed, May 23, 2012 at 3:03 AM, Kinuko Yasuda kin...@chromium.org wrote: Just to make sure, I assume 'the underlying storage' includes memory. Right. For simple Blobs without a mutable backing store, all of this essentially optimizes away. We should also make it clear whether .size and .lastModifiedDate should return live state or should just returning the same constant values. (I assume the latter) It would be the values at the time of the snapshot state. (I doubt it was ever actually intended that lastModifiedDate always return the file's latest mtime. We'll find out when one of the editors gets around to this thread...) I think the ideal behavior is that it reflects values at snapshot state, but that reads if snapshot state has modified fail. -- A*
Re: [File API] Blob URI creation
On May 30, 2012, at 6:48 PM, Glenn Maynard wrote: On your main question, I've had the same thought in the past--a url property on Blob which simply creates a new auto-revoking blob URL. I didn't bring it up since I'm not sure if creating a URL for a blob is actually something you do so often that it's worth having a shortcut. If so, a function is probably better than a property--more future-proof, and it'd be unusual on the platform to have a property that returns a different value every time you read it. On Wed, May 30, 2012 at 1:50 PM, Rich Tibbett ri...@opera.com wrote: Yes, this might be a better solution. I was working on what was available in the editor's draft and looking for a way to remove the need to ever call revokeObjectUrl. This is part of what's wrong with oneTimeOnly--it *doesn't* actually completely remove the need to call revokeObjectUrl. For example: function f(blob) { var url = URL.createObjectURL(blob, {oneTimeOnly: true}); if(showImage) img.src = url; else URL.revokeObjectURL(url); } Without the revoke case, the URL (and so the whole blob) is leaked as it's never actually used. autoRevoke doesn't have this problem. Arun/Jonas: Can we hide this feature in the spec before more people implement it, or at least tag it with not ready for implementations or something? I'll do one better, and introduce autoRevoke semantics: http://www.w3.org/TR/2012/WD-FileAPI-20120712/#creating-revoking By default, this does not need a corresponding revokeObjectURL() call. In order for Blob URLs to persist past a stable state (for that unit of script) createObjectURL has to be invoked with autoRevoke set to false. That is, you shouldn't ever have to pass a Blob URI obtained via Blob.getURL through revokeObjectUrl because it assumes some auto-revocation behavior. Using microtasks to release at the next stable state does seem ok as long as developers have a very good understanding of when a Blob URI will be implicitly revoked. Saying that you can use a Blob URI exactly once, as per onetimeonly could still end up being easier to understand though. (s/microtasks/stable states/; they're not quite the same) It's actually a bit hard to understand (or easy to misunderstand), since there's no clear concept of using a URL. For example, if you start two XHR's on the same URL one after the other in the same script, the order in which the fetches actually begin is undefined (they happen in different task queues), so which would succeed is undefined. (Some work would also be needed here for the autoRevoke approach, but it's much simpler.) autoRevoke is pretty simple from the user's perspective: the URL is revoked when your script returns to the browser. In fact, I think this addresses the lion's share of use cases, and if a developer wants to explicitly create longer lasting Blob URLs, they have that option (just not by default). -- A*
Re: BlobBuilder.append() should take ArrayBufferView in addition to ArrayBuffer
On Apr 27, 2012, at 1:28 AM, Anne van Kesteren wrote: On Fri, 27 Apr 2012 00:13:42 +0200, Arun Ranganathan aranganat...@mozilla.com wrote: The constructor will switch to use ArrayBufferView in lieu of ArrayBuffer, but the read method exposed on FileReader and FileReaderSync will read files into memory as ArrayBuffers. Since the constructor is not widely deployed yet would it not be better to stick to ArrayBufferView exclusively, just like XMLHttpRequest.send()? The Blob constructor will already use ArrayBufferView. Above, do you mean, have a method of the sort FileReader.readAsArrayBufferView or something like that? *.readAsArrayBuffer is already deployed, and it might be weird to force a view on an ArrayBuffer when read into memory. -- A* -- Anne van Kesteren http://annevankesteren.nl/
Re: BlobBuilder.append() should take ArrayBufferView in addition to ArrayBuffer
On Apr 24, 2012, at 7:00 PM, David Herman wrote: On Apr 24, 2012, at 3:53 PM, David Herman wrote: On Apr 12, 2012, at 2:48 PM, Arun Ranganathan wrote: I intend to add ArrayBufferView as a parameter to the Blob constructor . Would it be possible also to allow passing an ArrayBuffer with an offset and length as an additional alternative? This would eliminate the need to create a view when you just want a slice of the buffer but don't particularly want/need to think of it as a Uint8/Uint32/whatever array. Oh, I just saw in the bug that the idea is to *eliminate* the ArrayBuffer argument option entirely. If that's the case then disregard my question. The constructor will switch to use ArrayBufferView in lieu of ArrayBuffer, but the read method exposed on FileReader and FileReaderSync will read files into memory as ArrayBuffers. -- A*
Re: Delay in File * spec publications in /TR/ [Was: CfC: publish LCWD of File API; deadline March 3]
On Mar 30, 2012, at 2:25 PM, ext Eric U wrote: On Fri, Mar 30, 2012 at 5:39 AM, Arthur Barstow art.bars...@nokia.com wrote: Hi All - the publication of the File API LC was delayed because of some logistical issues for Arun as well as some additional edits he intends to make. This delay also resulted in Eric's two File * specs not being published since they have a dependency on the latest File API spec. Arun - can you please give us at least a rough idea when you expect the spec to be ready for LC publication? Jonas - as co-Editor of File API, can you help get the File API LC published? Eric - your File * docs were last published in April 2011 so I think it would be good to get new versions published in /TR/ soon-ish. OTOH, if they have dependencies on the latest File API, it may be better to postpone their publication until File API is published. WDYT? If it's going to be more than a month to get Arun+Jonas's spec up, we might as well go ahead and publish mine; they've had quite a bit of change. If it's less than that, let's just do them all together. Eric - I haven't received any responses from Arun or Jonas so I will contact you off list about getting your two File specs ready for publication on /TR/. Apologies for the delay in getting back to the listserv about changes to File API. Internal logistics have been resolved, and I look forward to making edits on the File API specification. As Eric points out, it is probably going to be more than 2 weeks till I can account for all the necessary edits. Eric's drafts don't need to be held up this timeline, though we should probably keep some things closely in sync., especially events, errors, etc. -- A*
Re: [FileAPI] Deterministic release of Blob proposal
Feras, In practice, I think this is important enough and manageable enough to include in the spec., and I'm willing to slow the train down if necessary, but I'd like to understand a few things first. Below: - Original Message - At TPAC we discussed the ability to deterministically close blobs with a few others. As we’ve discussed in the createObjectURL thread[1], a Blob may represent an expensive resource (eg. expensive in terms of memory, battery, or disk space). At present there is no way for an application to deterministically release the resource backing the Blob. Instead, an application must rely on the resource being cleaned up through a non-deterministic garbage collector once all references have been released. We have found that not having a way to deterministically release the resource causes a performance impact for a certain class of applications, and is especially important for mobile applications or devices with more limited resources. In particular, we’ve seen this become a problem for media intensive applications which interact with a large number of expensive blobs. For example, a gallery application may want to cycle through displaying many large images downloaded through websockets, and without a deterministic way to immediately release the reference to each image Blob, can easily begin to consume vast amounts of resources before the garbage collector is executed. To address this issue, we propose that a close method be added to the Blob interface. When called, the close method should release the underlying resource of the Blob, and future operations on the Blob will return a new erro r , a ClosedError. This allows an application to signal when it's finished using the Blob. Do you agree that Transferable (http://dev.w3.org/html5/spec/Overview.html#transferable-objects) seems to be what we're looking for, and that Blob should implement Transferable? Transferable addresses the use case of copying across threads, and neuters the source object (though honestly, the word neuter makes me wince -- naming is a problem on the web). We can have a more generic method on Transferable that serves our purpose here, rather than *.close(), and Blob can avail of that. This is something we can work out with HTML, and might be the right thing to do for the platform (although this creates something to think about for MessagePort and for ArrayBuffer, which also implement Transferable). I agree with your changes, but am confused by some edge cases: To support this change, the following changes in the File API spec are needed: * In section 6 (The Blob Interface) - Addition of a close method. When called, the close method releases the underlying resource of the Blob. Close renders the blob invalid, and further operations such as URL.createObjectURL or the FileReader read methods on the closed blob will fail and return a ClosedError. If there are any non-revoked URLs to the Blob, these URLs will continue to resolve until they have been revoked. - For the slice method, state that the returned Blob is a new Blob with its own lifetime semantics – calling close on the new Blob is independent of calling close on the original Blob. *In section 8 (The FIleReader Interface) - State the FileReader reads directly over the given Blob, and not a copy with an independent lifetime. * In section 10 (Errors and Exceptions) - Addition of a ClosedError. If the File or Blob has had the close method called, then for asynchronous read methods the error attribute MUST return a “ClosedError” DOMError and synchronous read methods MUST throw a ClosedError exception. * In section 11.8 (Creating and Revoking a Blob URI) - For createObjectURL – If this method is called with a closed Blob argument, then user agents must throw a ClosedError exception. Similarly to how slice() clones the initial Blob to return one with its own independent lifetime, the same notion will be needed in other APIs which conceptually clone the data – namely FormData, any place the Structured Clone Algorithm is used, and BlobBuilder. Similarly to how FileReader must act directly on the Blob’s data, the same notion will be needed in other APIs which must act on the data - namely XHR.send and WebSocket. These APIs will need to throw an error if called on a Blob that was closed and the resources are released. So Blob.slice() already presumes a new Blob, but I can certainly make this clearer. And I agree with the changes above, including the addition of something liked ClosedError (though I suppose this is an important enough error + exception to hash out with HTML and DOM4, and once again, the name is TBD). In your implementation, what happens exactly to Eric's edge cases, namely: xhr.send(blob); blob.close(); // method name TBD // AND frameRef.src = URL.createObjectURL(blob); blob.close() // method name TBD In my opinion, the