Re: Blob URLs | autoRevoke, defaults, and resolutions

[Michael Nordman]

Fwiw, to the extent it may be helpful when it comes to spec writing, here
are some quick-n-dirty thoughts about how some approximation of the
'autorevoke' behavior could be implemented in chromium.

1) Extend the lifetime of the PublicBlobURL registration until *after* the
last latchee has redeemed its ticket to ride the url. The url registration
itself is refcounted. The url registration implies the underlying data is
not reclaimable.  At time of coining the url registration, a microtask is
scheduled to release it. Latchee's may addref it safely prior to microtask
execution, keeping the url registration valid until a later balancing
release. This would be a non-compliant approximation of the way the spec is
shaping up... but good enough... maybe has the important
characteristics of freeing memory up eventually (generally prior to doc
unload) and guaranteeing a latchee's ticket to ride.

2) Extend the lifetime of the underlying BlobData until *after* the latchee
has redeemed its ticket to ride the url, but revoke the URL well in advance
of that. The url registration is not refcounted, The url registration
implies the underlying data is not reclaimable. There is a means to lookup
a handle to the data given the url.  The underlying blob data is
refcounted. Latchee's must take and release a ref on that. The url
registration is revoked at microtask execution time after being coined.
Piggy back a handle to the blob data on future Fetch (or other network
requests) for the PublicBlobURL. Ignore the URL when processing those
requests and refer only to the piggybacked blob data handle. Probably a
more compliant approach, but maybe more tedious to implement in chromium
(since the URL is no longer useful as the identifier for what to
fetch/load/whathaveyou, we need sideband data for that in addition to
addref/release at url latch/redemption times).

>  (I'm confused, because we've talked about this distinction several
times.)
lol, picture a herd of cats chasing their tails, you can call me calico :)


On Tue, May 7, 2013 at 1:34 PM, Glenn Maynard  wrote:

> On Tue, May 7, 2013 at 9:45 AM, Anne van Kesteren wrote:
>
>> On Mon, May 6, 2013 at 11:11 PM, Jonas Sicking  wrote:
>> > The only thing that's different about XHR is that the first step in my
>> > list lives in one function, and the other steps live in another
>> > function. Doesn't seem to have any effect on the discussions here
>> > other than that we'd need to define which of the two functions does
>> > the step which grabs a reference to the Blob.
>>
>> Fair enough. So I guess we can indeed fix this by changing
>> http://fetch.spec.whatwg.org/#concept-fetch to get a reference to the
>> Blob/MediaStream/... before returning early as Arun suggested.
>>
>
> Step 1 is resolve, step 3 is fetch.  Moving it into step 1 means it would
> go in resolve, not fetch.  Putting it in fetch wouldn't help, since fetch
> doesn't always start synchronously.  (I'm confused, because we've talked
> about this distinction several times.)
>
> --
> Glenn Maynard
>
>

Re: IndexedDB events for object storage add, put and delete

[Michael Nordman]

This could be accomplished with a separate pub/sub capability, instead of
complicating the IDB interfaces and allocating new functions to that
component.

We (chromium project) have open bug about a "broadcastMessage" function.
https://code.google.com/p/chromium/issues/detail?id=161070


On Tue, Feb 5, 2013 at 2:21 PM, pira...@gmail.com  wrote:

> This should also be added, then...
>
> 2013/2/5 Dale Harvey :
> > They can, I was just saying that they wont do that by default (as I
> assume a
> > native implementation would), you need to write your own messaging system
> > out of band
> >
> > Cheers
> > Dale
> >
> >
> > On 5 February 2013 22:12, pira...@gmail.com  wrote:
> >>
> >> Why it can propagate over tabs if all of them are accessing to the
> >> same database?
> >>
> >> 2013/2/5 Dale Harvey :
> >> > The problem with emitting change notification on writes is that they
> >> > dont
> >> > propogate across tabs, my library has to use localstorage to emit
> events
> >> > across tabls and keep track of a change sequence in each tab
> >> >
> >> > This would be a welcome addition to the spec (after we get to
> enumerate
> >> > databases) :)
> >> >
> >> >
> >> > On 5 February 2013 21:59, pira...@gmail.com 
> wrote:
> >> >>
> >> >> One solution would be to don't call directly to IndexedDB methods but
> >> >> instead use custom wrappers that fit better with your application
> >> >> (this is what I'm doing), but definitelly I totally agree with you
> >> >> that IndexedDB should raise events when a row has been
> >> >> inserted/updated/deleted. I think it was talked about it would be an
> >> >> explosion of events, but I'm not sure about this... having events
> >> >> would be useful to develop triggers to maintain database consistency,
> >> >> for example :-)
> >> >>
> >> >> 2013/2/5 Miko Nieminen :
> >> >> > Hi,
> >> >> >
> >> >> > I'm new to this forum and I'm not completely sure if I'm posting to
> >> >> > right
> >> >> > list. I hope I am.
> >> >> >
> >> >> > I've been playing with IndexedDB to learn how to use it and around
> >> >> > this
> >> >> > experiment I wrote a blog article about my experiences.
> >> >> >
> >> >> > While writing my article, I realized there is no way to add event
> >> >> > listeners
> >> >> > for object store to get notifications when new object is added,
> >> >> > existing
> >> >> > one
> >> >> > is modified or one is deleted. I think lack of these events makes
> >> >> > some
> >> >> > use
> >> >> > cases much more complicated than one would hope. Use cases like
> >> >> > keeping
> >> >> > local data in sync with remote database, synchronizing views
> between
> >> >> > multiple windows or creating generic data indexers or manipulation
> >> >> > libraries. I know there are ways to go around the lack of these
> >> >> > events,
> >> >> > but
> >> >> > having those would make things much easier.
> >> >> >
> >> >> > Is there any reason why these are not included in the
> specification?
> >> >> > It
> >> >> > just
> >> >> > feels bit strange when similar mechanism is included in WebStorage
> >> >> > API,
> >> >> > but
> >> >> > not in IDB. I suppose right moment to emit these events would be
> just
> >> >> > after
> >> >> > emitting transaction complete.
> >> >> >
> >> >> > I wasn't able to find any references from the archives and I hope
> I'm
> >> >> > not
> >> >> > asking same question again. Also I hope I'm not asking this
> question
> >> >> > too
> >> >> > late.
> >> >> >
> >> >> > My blog article talks about this in a bit more detailed level under
> >> >> > header
> >> >> > "Shortcomings of IndexedDB".. The whole article is quite long so
> you
> >> >> > might
> >> >> > want to skip most of it. You can find it from
> >> >> >
> >> >> >
> >> >> >
> http://mini-thinking.blogspot.co.uk/2013/02/web-app-example-using-indexeddb.html
> >> >> >
> >> >> > Thanks,
> >> >> > --
> >> >> > Miko Nieminen
> >> >> > miko.niemi...@iki.fi
> >> >> > miko.niemi...@gmail.com
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> "Si quieres viajar alrededor del mundo y ser invitado a hablar en un
> >> >> monton de sitios diferentes, simplemente escribe un sistema operativo
> >> >> Unix."
> >> >> – Linus Tordvals, creador del sistema operativo Linux
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> "Si quieres viajar alrededor del mundo y ser invitado a hablar en un
> >> monton de sitios diferentes, simplemente escribe un sistema operativo
> >> Unix."
> >> – Linus Tordvals, creador del sistema operativo Linux
> >
> >
>
>
>
> --
> "Si quieres viajar alrededor del mundo y ser invitado a hablar en un
> monton de sitios diferentes, simplemente escribe un sistema operativo
> Unix."
> – Linus Tordvals, creador del sistema operativo Linux
>
>

Re: Making offline apps easier?

[Michael Nordman]

> Making offline apps easier?

Feels like we're still at the 'make it possible' phase more so than the
'make it easier' phase.

I think it's fitting that appcache and widgets were listed at opposite ends
in your list because  the intent of the former is to make it possible for
"drive-by-web" sites to function without a network, while the intent of the
latter is to provide an html+css+js packaging format to facilitate
distribution and deployment out-of-band of the traditional drive-by-web
experience. At least imho.

There are a number of vendor specific packaging formats being heatedly
worked on as well as the ecosystems to distribute and deploy them. In part
at least, the rapid rate of change in the packaged app world(s) is possible
due to the lack of standards. Different ecosystems  are coming into being
independent of one another. Imposing a standards process would probably
slow their progress which probably explains the lack of interest in vendors
on the "official" widget stack.

In contrast, there is less progress being made in direct support of the
"drive-by-web" case. There is the "fixit" group which is groping around the
the area. And some recent spec changes that might help, but I think vendors
are somewhat hesitant to make changes for a handful of reasons (in no
particular order):
* conflicts of interest with their vendor specific packaged apps formats
* FUD about fragmenting the web
* hazy vision on how to build upon appcache or some replacement

Given that view of the world, I'd vote for the public-webapps group to
focus on the "drive-by-web" case, because to step into the packaged app
world could hurt current developments more than it helps. And as with most
things that are developed for the "drive-by-web" case, the packaged app
worlds could pick them up more or less for free (by virtue of constructs
like the embedded http://something/on/the/drive-by-web";> tag).

Explicit storage apis like IndexedDB, Filesystem, and LocalStorage are very
necessary for "offline". Development of two of those three are humming
along nicely in a critical mass of browsers. The real laggard is the
appcache. We need a good way to bootstrap apps while offline. That can be
accomplished to a degree but it's clunky and comes with some odd
constraints and baggage.

Imo, a critically missing piece of the puzzle is how to accommodate deep
url  navigation w/o a network.  Cold start the browser, navigate via
bookmark or a history enabled auto-complete to a  url buried deep in the
"app".  We have no good story for that.  The ability to store and retrieve
data on the local system is covered already. Some amount of HTTP resource
caching is done already. But very little is done that allows intelligent
use of those stashes of locally stored data *before* a document is loaded
into a frame.

Another missing piece of the puzzle is providing better control over the
set of HTTP resources that are cached. The flat list of urls in the
manifest file is too constraining. Some means of grouping subsets
of resources for addition, update, and removal.

The automagic addition of "master" entries is too often an unwelcome,
annoying, surprising addition.  Just got another bug report about that
today.

Most difficult is finding a way to craft things such that the "online"  vs
"offline" experiences aren't completely different from both the end user
and developer point of view.

I think the best way to get to a satisfying answer for the critically
missing piece  is by allowing scripts to satisfy resource requests,
computing/composing a response from locally stored data, or designating the
return of a cached HTTP response, or trying to make a network request first
then taking local action. I'm less sure that hanging that capability off of
the appcache is the best way to go, but it seems like an option. Or that
capability could be a specified as separate piece. I also think a
capability like this would give app developers the tool they really need to
blur the distinction between 'online' vs 'offline' in ways that make sense
for their apps.

If public-webapps wants to help make offline easier, the appcache feature
set is the area that needs the most help.

Cheers
Michael



On Mon, Nov 12, 2012 at 8:11 AM, Charles McCathie Nevile <
cha...@yandex-team.ru> wrote:

> Hi folks
>
> We have a handful of things that could be on our plate here.
>
> 1. Appcache.
> Yeah, we know. But if nobody makes a proposal, nothing will get better.
> (There is also a "fixing-appcache" community group taht might come here
> with proposals).
> 2. Offline apps breakout at TPAC
> http://www.w3.org/wiki/**TPAC2012/Offline_AppsAshok
>  Malhotra (Oracle, who
> aren't webapps members), ran a session. Where we agreed to divide the
> thoughts into "the app shell" and "the app data" - either of which could
> be brought from offline, or online, independently. Plus I made a half-page
> note about different ways of having apps offline
> 3. Fi

Re: Pre-fetch rough draft

[Michael Nordman]

The appcache is encumbered with guarantees about atomically updating a set
of resources and then explicitly not hitting the network for them once up
to date to ensure the site/app will function if the network really is
complete gone.

This gist of this prefetch list seems different. More of a hint to warm
things up if possible and viola things are more responsive if those hints
are taken.



On Tue, Oct 30, 2012 at 2:58 PM, Mounir Lamouri  wrote:

> On 10/30/2012 10:22 AM, Charles McCathieNevile wrote:
> > Hi,
> >
> > I mentioned this and it's somethign we are working on.
> >
> > Basic idea: site provides list of resources that it uses and can be
> > cached for general improvements on the whole site. (We're seeing
> > load-time improvement from 50% - 300% in our testing. We are using it on
> > sites - mail.yandex.ru/prefetch.txt has an example).
> >
> > The draft "spec" here is still very rough, but it shows what we've
> > implemented and some of what we think it is good for.
> >
> > This is meant as input to the appcache/packaging/etc discussion, and may
> > or may not be something this group takes on.
>
> At a first glance, it seems that appcache could do the same things.
> Which use cases appcache can't currently solve? In the process of fixing
> appcache, it could be interesting to add those use cases.
>
> --
> Mounir
>
>

Re: Lazy Blob

[Michael Nordman]

The URLObject proposal is a pretty slick way of cooking up a request in
contextA for later (and all manner of) retrieval in contextB.


On Thu, Aug 2, 2012 at 4:23 PM, Glenn Maynard  wrote:

> I'd suggest the following.
>
> - Introduce an interface URLObject (bikeshedding can come later), with no
> methods.  This object is supported by structured clone.
> - Add XMLHttpRequest.getURLObject(optional data), which returns a new
> URLObject.  This can only be called while XMLHttpRequest is in the OPENED
> state.  The returned URLObject represents the resource that would have been
> fetched if xhr.send() had been called.  No XHR callbacks are performed, and
> no network activity takes place.  The "data" argument acts like send(data),
> to specify the request entity body that will be sent.
> - Adjust URL.createObjectURL to take (Blob or URLObject), which returns a
> URL that works the same way as Blob URLs.
>
> Example:
>
>
> var xhr = new XMLHttpRequest();
> xhr.open("GET", "resource.jpg");
> var urlObject = xhr.getURLObject();
> var newURL = URL.getObjectURL(urlObject);
> img.src = newURL;
>
> Some benefits:
>
> - We inherit XHR's ability to manipulate the request (eg.
> setRequestHeader), and all the security considerations that have gone into
> it.
> - The result is very much like Blob: you can transfer it around where you
> want it (workers or other pages), create object URLs, and then use those
> URLs with every other API (including Web Sockets, incidentally), since
> they're just like blob URLs.
> - It avoids a huge pitfall of "getBlobFromURL", by not implementing a
> whole new mechanism for specifying a request.  We already have XHR.
> - It avoids the pitfalls of returning a Blob: we don't make any network
> request at all at creation time to learn the size (avoiding spamming
> hundreds of HEAD requests), and it won't run into problems with servers
> with broken HEAD, transfer encodings, and so on.
>
> Any fetches performed as a result of this would be done under the origin
> of the page that created it, even if you transfer a URLObject somewhere
> else.  This seems safe, since the caller can do nothing with it except
> trigger a fetch as-is, but it would need careful security review (as would
> any API like this).  Passing one of these URLs back to XHR would need extra
> consideration (eg. should you be able to specify more headers?).
>
> (Note that I've spent some time thinking about this because I think it's
> technically interesting, but I haven't looked over the use cases closely
> enough to say whether I think it'd be worthwhile or not.)
>
> --
> Glenn Maynard
>
>

Re: Lazy Blob

[Michael Nordman]

Maybe another XHR based way to phrase this: define a new response type for
XHR that results in the construction of a lazyBlob. I guess that's similar
to the more explicit xhr.makeLazyBlob() method, but allows its construction
to be async, and for greater reuse of the same signaling for progress and
errors, and to capture the results of a POST as a lazyBlob as well.

Knowing the length and to some degree the content-type upfront is trouble.
An xhr.responseType of 'stream' would have some trouble with the
content-type too.


On Wed, Aug 1, 2012 at 10:57 AM, Charles Pritchard  wrote:

> On Aug 1, 2012, at 8:44 AM, Glenn Maynard  wrote:
>
> On Wed, Aug 1, 2012 at 9:59 AM, Robin Berjon  wrote:
>
>> var bb = new BlobBuilder()
>
> ,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";, "GET",
>> { Authorization: "Basic DEADBEEF" });
>>
>> Everything is the same as the previous version but the method and some
>> headers can be set by enumerating the Object. I *think* that those are all
>> that would ever be needed.
>>
>
> We already have an API to allow scripts to make network requests: XHR.
>  Please don't create a new API that will end up duplicating all of that.
>  However this might be done, it should hang off of XHR.
>
> Ideally, a new responseType could be added to XHR which operates like
> "blob", except instead of reading the whole resource, it just performs a
> HEAD to retrieve the response length and immediately returns the Blob,
> which can be read to perform further Content-Range reads.  This sits
> relatively cleanly within the XHR API.  It also has nice security
> properties, as if you send that Blob somewhere else--possibly to a
> different origin--it can do nothing with the Blob but read it as it was
> given.
>
> The trouble is chunked responses, where the length of the resource isn't
> known in advance, since Blobs have a static length and aren't designed for
> streaming.
>
>
>
> The "HEAD" request seems a little like a FileEntry for use with FileReader.
>
> Something similar (very similar) is done for http+fuse on several
> platforms.
>
> It's quite useful for working with large archives such as PDF or ZIP.
>
>
> -Charles
>

Re: Transferable and structured clones, was: Re: [FileAPI] Deterministic release of Blob proposal

[Michael Nordman]

>> Separately defining blobs to be transferrable feels like an unneeded
>> complexity. A caller wishing to
>> neuter after sending can explicit call .close() rather than relying on
>> more obscure artifacts of having also put the 'blob' in a
>> 'transferrable' array.
>
>
> You can always call close() yourself, but Blob.close() should use the
> "neuter" mechanism already there, not make up a new one.

Blobs aren't transferable, there is no existing mechanism that applies
to them. Adding a blob.close() method is independent of making blob's
transferable, the former is not prerequisite on the latter.

Re: Transferable and structured clones, was: Re: [FileAPI] Deterministic release of Blob proposal

[Michael Nordman]

Sounds like there's a good case for an explicit blob.close() method
independent of 'transferable'. Separately defining blobs to be
transferrable feels like an unneeded complexity. A caller wishing to
neuter after sending can explicit call .close() rather than relying on
more obscure artifacts of having also put the 'blob' in a
'transferrable' array.

On Tue, Mar 6, 2012 at 1:18 PM, Kenneth Russell  wrote:
> On Tue, Mar 6, 2012 at 12:04 PM, Greg Billock  wrote:
>> On Mon, Mar 5, 2012 at 6:46 PM, Charles Pritchard  wrote:
>>> On 3/5/2012 5:56 PM, Glenn Maynard wrote:
>>>
>>> On Mon, Mar 5, 2012 at 7:04 PM, Charles Pritchard  wrote:

 Do you see old behavior working something like the following?


 var blob = new Blob("my new big blob");
 var keepBlob = blob.slice(); destination.postMessage(blob, '*', [blob]);
 // is try/catch needed here?
>>>
>>>
>>> You don't need to do that.  If you don't want postMessage to transfer the
>>> blob, then simply don't include it in the transfer parameter, and it'll
>>> perform a normal structured clone.  postMessage behaves this way in part for
>>> backwards-compatibility: so exactly in cases like this, we can make Blob
>>> implement Transferable without breaking existing code.
>>>
>>> See http://dev.w3.org/html5/postmsg/#posting-messages and similar
>>> postMessage APIs.
>>>
>>>
>>> Web Intents won't have a transfer map argument.
>>> http://dvcs.w3.org/hg/web-intents/raw-file/tip/spec/Overview.html#widl-Intent-data
>>>
>>> For the Web Intents structured cloning algorithm, Web Intents would be
>>> inserting into step 3:
>>>     If input is a Transferable object, add it to the transfer map.
>>> http://www.whatwg.org/specs/web-apps/current-work/multipage/common-dom-interfaces.html#internal-structured-cloning-algorithm
>>>
>>> Then Web Intents would move the first section of the structured cloning
>>> algorithm to follow the internal cloning algorithm section, swapping their
>>> order.
>>> http://www.whatwg.org/specs/web-apps/current-work/multipage/common-dom-interfaces.html#safe-passing-of-structured-data
>>>
>>> That's my understanding.
>>
>> We've been discussing the merits of this approach vs using a transfer
>> array argument. There's a lot to like about this alternative -- it
>> conserves arguments and looks simpler than the transfer map, as well
>> as not having the headaches of whether you can do (null, [port]) or
>> (port, [port]) and concerns like that.
>>
>> The advantage of using the transfer map param is that it is more
>> contiguous with existing practice. We'd kind of hoped that this
>> particular debate was finalized before we got to the point of needing
>> to make a decision, so we bluffed and left it out of the web intents
>> spec draft. :-) At this point, I'm leaning toward needing to add a
>> transfer map parameter, and then dealing with that alongside other
>> uses, given the state of thinking on Transferables support and the
>> need to make this pretty consistent across structure clone
>> invocations.
>>
>> I do think that complexity might be better solved by the type system
>> (i.e. a "new Transferable(ArrayBuffer)"), which would require a
>> different developer mechanic to set up clone vs transfer, but would
>> relieve complexity in the invocation of structured clone itself:
>> transferables could just always transfer transparently. I don't know
>> if, given current practice with MessagePort, that kind of solution is
>> available.
>
> A change like this would be feasible as long as it doesn't break
> compatibility. In other words, the current Transferable array would
> still need to be supported, but Transferable instances (or perhaps
> instances of some other type) wrapping another Transferable object
> would also express the intent.
>
> The current API for Transferable and postMessage was informed by the
> realization that the previous sequence argument to
> postMessage was essentially already expressing the Transferable
> concept.
>
> I'm not familiar with the Web Intents API, but at first glance it
> seems feasible to overload the constructor, postResult and postFailure
> methods to support passing a sequence as the last
> argument. This would make the API look more like postMessage and avoid
> adding more transfer semantics. Is that possible?
>
>
>>> Something like this may be necessary if Blob were a Transferable:
>>> var keepBlob = blob.slice();
>>> var intent = new Intent("-x-my-intent", blob);
>>> navigator.startActivity(intent, callback);
>>>
 And we might have an error on postMessage stashing it in the transfer
 array if it's not a Transferable on an older browser.
>>>
>>>
>>>
>>>
>>> Example of how easy the neutered concept applies to Transferrable:
>>>
>>> var blob = new Blob("my big blob");
>>> blob.close();
>>>
>>>
>>> I like the idea of having Blob implement Transferrable and adding close to
>>> the Transferrable interface.
>>> File.close could have a better relationship with the cache and/or locks on
>>

Re: [FileAPI] Length of the opaque string for blob URLs

[Michael Nordman]

There is no requirement for that in the spec/draft, it's useful for our
implementation.

On Fri, Dec 16, 2011 at 3:06 PM, Charles Pritchard  wrote:

> Is there something requiring that the origin be part of the URL?
>
>
>
> On Dec 16, 2011, at 2:29 PM, Michael Nordman  wrote:
>
> > "and MUST be at least 36 characters long"
>
> I can't think of any reason for that requirement, seems fine to delete it.
>
> Webkit and Chrome do use guids but also embed the origin in these url.
>
>
> On Fri, Dec 16, 2011 at 5:58 AM, Jarred Nicholls < 
> jar...@webkit.org> wrote:
>
>> On Fri, Dec 16, 2011 at 6:27 AM, Anne van Kesteren < 
>> ann...@opera.com> wrote:
>>
>>> On Fri, 16 Dec 2011 12:21:34 +0100, Arun Ranganathan 
>>> <
>>> aranganat...@mozilla.com> wrote:
>>>
>>>> Adrian: I'm willing to relax this.  I suppose it *is* inconsistent to
>>>> insist on 36 chars when we don't insist on UUID.  But I suspect when it
>>>> comes time to making blob: a registered protocol (it was discussed on the
>>>> IETF/URI listserv), the lack of MUSTs will be a sticking point.  We'll take
>>>> that as it comes, though :)
>>>>
>>>
>>> I do not really see why Chrome cannot simply use UUID as well. It's not
>>> exactly rocket science. It seems that is the only sticking point to just
>>> having the same type of URLs across the board.
>>
>>
>> The consistency and predictability of having UUIDs across the board could
>> prove useful.
>>
>>
>>>
>>>
>>>
>>> --
>>> Anne van Kesteren
>>>  <http://annevankesteren.nl/>http://annevankesteren.nl/
>>>
>>>
>>
>

Re: [FileAPI] Length of the opaque string for blob URLs

[Michael Nordman]

> "and MUST be at least 36 characters long"

I can't think of any reason for that requirement, seems fine to delete it.

Webkit and Chrome do use guids but also embed the origin in these url.


On Fri, Dec 16, 2011 at 5:58 AM, Jarred Nicholls  wrote:

> On Fri, Dec 16, 2011 at 6:27 AM, Anne van Kesteren wrote:
>
>> On Fri, 16 Dec 2011 12:21:34 +0100, Arun Ranganathan <
>> aranganat...@mozilla.com> wrote:
>>
>>> Adrian: I'm willing to relax this.  I suppose it *is* inconsistent to
>>> insist on 36 chars when we don't insist on UUID.  But I suspect when it
>>> comes time to making blob: a registered protocol (it was discussed on the
>>> IETF/URI listserv), the lack of MUSTs will be a sticking point.  We'll take
>>> that as it comes, though :)
>>>
>>
>> I do not really see why Chrome cannot simply use UUID as well. It's not
>> exactly rocket science. It seems that is the only sticking point to just
>> having the same type of URLs across the board.
>
>
> The consistency and predictability of having UUIDs across the board could
> prove useful.
>
>
>>
>>
>>
>> --
>> Anne van Kesteren
>> http://annevankesteren.nl/
>>
>>
>

Re: Is BlobBuilder needed?

[Michael Nordman]

This ultimately amounts to syntactic sugar compared to the existing api.
It's tasty, but adds no new functionality. Also there's still the issue of
how this new api would provide the existing functionality around line
endings, so less functionality at the moment. I'm not opposed to
additions/enhancements,  just want to put it in perspective and to question
whether the api churn is worth it.

On Mon, Oct 24, 2011 at 10:19 PM, Erik Arvidsson  wrote:

> On Mon, Oct 24, 2011 at 19:54, Jonas Sicking  wrote:
> > Sure. Though you could also just do
> >
> > var b = new Blob();
> > b = new Blob([b, data]);
> > b = new Blob([b, moreData]);
>
> That works for me.
>
> --
> erik
>
>

Re: Is BlobBuilder needed?

[Michael Nordman]

It's nice. Like Ojan said, I don't see this as mutually exlusive with
BlobBuilder either. There's nother peice of data to associate with a
blob... the content-type... that would need to be provided in the ctor.

On Mon, Oct 24, 2011 at 4:01 PM, Ojan Vafai  wrote:

> On Mon, Oct 24, 2011 at 3:52 PM, Jonas Sicking  wrote:
>
>> Hi everyone,
>>
>> It was pointed out to me on twitter that BlobBuilder can be replaced
>> with simply making Blob constructable. I.e. the following code:
>>
>> var bb = new BlobBuilder();
>> bb.append(blob1);
>> bb.append(blob2);
>> bb.append("some string");
>> bb.append(myArrayBuffer);
>> var b = bb.getBlob();
>>
>> would become
>>
>> b = new Blob([blob1, blob2, "some string", myArrayBuffer]);
>>
>
> I like this API. I think we should add it regardless of whether we get rid
> of BlobBuilder. I'd slightly prefer saying that Blob takes varargs and rely
> on ES6 fanciness to expand the array into varargs.
>
> In theory, a BlobBuilder could be backed by a file on disk, no? The
> advantage is that if you're building something very large, you don't
> necessarily need to be using all that memory. You can imagine a UA having
> Blobs be fully in-memory until they cross some size threshold.
>
>
>> or look at it another way:
>>
>> var x = new BlobBuilder();
>> becomes
>> var x = [];
>>
>> x.append(y);
>> becomes
>> x.push(y);
>>
>> var b = x.getBlob();
>> becomes
>> var b = new Blob(x);
>>
>> So at worst there is a one-to-one mapping in code required to simply
>> have |new Blob|. At best it requires much fewer lines if the page has
>> several parts available at once.
>>
>> And we'd save a whole class since Blobs already exist.
>>
>> / Jonas
>>
>>
>

Re: [IndexedDB] transaction order

[Michael Nordman]

The behavior Israel describes is the behavior that I would expect as a
developer.

On Fri, Oct 14, 2011 at 1:51 PM, Israel Hilerio wrote:

> On Friday, October 07, 2011 4:35 PM, Israel Hilerio wrote:
> > On Friday, October 07, 2011 2:52 PM, Jonas Sicking wrote:
> > > Hi All,
> > >
> > > There is one edge case regarding transaction scheduling that we'd like
> > > to get clarified.
> > >
> > > As the spec is written, it's clear what the following code should do:
> > >
> > > trans1 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > > trans1.objectStore("foo").put("value 1", "mykey");
> > > trans2 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > > trans2.objectStore("foo").put("value 2", "mykey");
> > >
> > > In this example it's clear that the implementation should first run
> > > trans1 which will put the value "value 1" in object store "foo" at key
> > > "mykey". The implementation should then run trans2 which will write
> > > overwrite the same value with "value 2". The end result is that "value
> > > 2" is the value that lives in the object store.
> > >
> > > Note that in this case it's not at all ambiguous which transaction runs
> first.
> > > Since the two transactions have overlapping scope, trans2 won't even
> > > start until trans1 is committed. Even if we made the code something
> like:
> > >
> > > trans1 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > > trans1.objectStore("foo").put("value 1", "mykey");
> > > trans2 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > > trans2.objectStore("foo").put("value 2", "mykey");
> > > trans1.objectStore("foo").put("value 3", "mykey");
> > >
> > > we'd get the same result. Both put requests placed against trans1 will
> > > run first while trans2 is waiting for trans1 to commit before it
> > > begins running since they have overlapping scopes.
> > >
> > > However, consider the following example:
> > > trans1 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > > trans2 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > > trans2.objectStore("foo").put("value 2", "mykey");
> > > trans1.objectStore("foo").put("value 1", "mykey");
> > >
> > > In this case, while trans1 is created first, no requests are placed
> > > against it, and so no database operations are started. The first
> > > database operation that is requested is one placed against trans2. In
> > > the firefox implementation, this makes trans2 run before trans1. I.e.
> > > we schedule transactions when the first request is placed against
> > > them, and not when the IDBDatabase.transaction() function returns.
> > >
> > > The advantage of firefox approach is obvious in code like this:
> > >
> > > someElement.onclick = function() {
> > >   trans1 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > >   ...
> > >   trans2 = db.transaction(["foo"], IDBTransaction.READ_WRITE);
> > >   trans2.objectStore.put("some value", "mykey");
> > >   callExpensiveFunction();
> > > }
> > >
> > > In this example no requests are placed against trans1. However since
> > > trans1 is supposed to run before trans2 does, we can't send off any
> > > work to the database at the time when the .put call happens since we
> > > don't yet know if there will be requests placed against trans1. Only
> > > once we return to the event loop at the end of the onclick handler will
> > trans1 be "committed"
> > > and the requests in trans2 can be sent to the database.
> > >
> > > However, the downside with firefox approach is that it's harder for
> > > applications to control which order transactions are run. Consider for
> > > example a program is parsing a big hunk of binary data. Before
> > > parsing, the program starts two transactions, one READ_WRITE and one
> > > READ_ONLY. As the binary data is interpreted, the program issues write
> > > requests against the READ_WRITE transactions and read requests against
> > > the READ_ONLY transaction. The idea being that the read requests will
> > > always run after the write requests to read from database after all
> > > the parsed data has been written. In this setup the firefox approach
> > > isn't as good since it's less predictable which transaction will run
> > > first as it might depend on the binary data being parsed. Of course,
> > > you could force the writing transaction to run first by placing a
> request
> > against it after it has been created.
> > >
> > > I am however not able to think of any concrete examples of the above
> > > binary data structure that would require this setup.
> > >
> > > So the question is, which solution do you think we should go with. One
> > > thing to remember is that there is a very small difference between the
> > > two approaches here. It only makes a difference in edge cases. The
> > > edge case being that a transaction is created, but no requests are
> > > placed against it until another transaction, with overlapping scope, is
> > created.
> > >
> > > Firefox approach has strictly better performance in th

Re: Storage Quota API

[Michael Nordman]

On Tue, Sep 27, 2011 at 1:26 PM, Charles Pritchard  wrote:

> On 9/27/2011 9:43 AM, Jonas Sicking wrote:
>
>> On Tue, Sep 27, 2011 at 6:12 AM, Kinuko Yasuda
>>  wrote:
>>
>>> Just to confirm:  Yes the interfaces are vendor prefixed (WebKit), and
>>> WebSQL, AppCache, IDB are treated as temporary in the current chromium
>>> implementation.
>>> On Tue, Sep 27, 2011 at 8:53 AM, Charles Pritchard
>>>  wrote:
>>>
 Any ideas on how to express temp v. Perm to IndexedDB?

>>> IIRC there's been a proposal to have a way to hint that an IndexedDB
>>> ObjectStore is 'evictable' or not:
>>> http://www.w3.org/Bugs/Public/**show_bug.cgi?id=11350
>>> Though it seems to be put off until later (version 2), I was assuming
>>> that
>>> once we have the 'evictable' option it would indicate the data's
>>> Temp/Perm
>>> attribute.
>>> Other storage APIs do not have a way to express temp/perm either.
>>>  Chromium's current policy is defaulting to conservative or less
>>> astonishment to the users (in our belief), so that they won't see
>>> unexpected
>>> prompts or unknown data pressing their disk space.
>>>
>> And instead getting unexpected data loss :)
>>
>
> What does happen in Chrome? From what I've seen, the AppData directory
> continues to balloon.
>

It's pretty primitive right now. There's a pool of TEMP storage that's
shared between all origins. A single origin can consume no more than 20% of
that pool. As usage approaches the limit, chrome will reclaim space consumed
by the LRU origins. The eviction grain size is an origin. We're adding
browser settings UI so users can control the TEMP pool size (if they
desire).

There's an additional mechanims avialable to 'chrome applications'.  In the
chrome-app manifest file, and app can say "unlimitedStorage". The storage
associated with those apps is not counted against the TEMP pool, and is not
subject to eviction, it's as if some app outside of the browser is using the
diskspace. This mechanism was put in place prior to us having the TEMP
management system. As TEMP management gets more sophisticated, i hope we can
phase out the 'unlimited' thing.


> I had an old SSD, about 32Gigs: Chrome is pretty liberal in its use of disk
> space. Short of the user
> clearing their history, I've not quite understood when and how the
> temporary caches vacate.
>
> Now that we are targeting Chromebooks, the "unexpected data loss" scenario
> is something
> that's getting baked into the application ux.
>
> -Charles
>
>

Re: Storage Quota API

[Michael Nordman]

On Mon, Sep 26, 2011 at 4:53 PM, Charles Pritchard  wrote:

> The callback style is prevalent in the File API, as well as IndexedDB. It
> seems quite fitting to me. Am I missing something?
>
> They are using vendor prefixing (WebKit).
>
> From what I read, WebSQL is temporary: I've not confirmed this.
>
> I'm super surprised that Chrome treats IDB as temporary. That seems like a
> major WebKit bug.
>

The storage policy is not a function of webkit. Chrome policies are distinct
from Safari or any other webkit based browsers.


>
> The enum style is also borrowed from
> FileSystem. I'd rather have strings too, but I've accepted the 0/1 enum
> values from File API as the way things are going to be.
>
> Any ideas on how to express temp v. Perm to IndexedDB?
>

There is no way to express that for IDB, ditto for WebSQL or AppCache. The
only storage api in which Perm can be expressed is in FileSystem. It's
possible to make a way to say "perm" for other types. We feel pretty
strongly that there's value in being able to use these interfaces w/o
prompts and that's the motivation for the general TEMP policy.


> AppCache is temp, I'm sure, though I'd want it as perm on installed web
> apps.
>
> -Charles
>
>
>
> On Sep 26, 2011, at 4:43 PM, Jonas Sicking  wrote:
>
> > Please don't use errorCallback/SuccessCallback. That's not used in any
> > other APIs that are part of the cross-browser web platform. Instead
> > return a request object on which events are fired.
> >
> > Don't use enums as the syntax sucks in JS. Use strings instead. We're
> > making the same transition in a lot of other APIs.
> >
> > There's also a problem in that Gecko treats IndexedDB as permanent
> > quota by default and Chrome treats it as temporary. Not sure how to
> > solve that problem.
> >
> > You guys have this prefixed in your implementation for now, right?
> >
> > / Jonas
> >
> > On Mon, Sep 26, 2011 at 2:49 PM, Charles Pritchard 
> wrote:
> >> Though unstable, Chromium via WebKit has introduced an API for working
> with
> >> storage quotas:
> >>
> >>
> https://groups.google.com/a/chromium.org/group/chromium-html5/msg/5261d24266ba4366
> >>
> >> In brief:
> >>
> >>  void queryUsageAndQuota(
> >>  unsigned short storageType,
> >>  optional StorageInfoUsageCallback successCallback,
> >>  optional StorageInfoErrorCallback errorCallback);
> >>  // Requests a new quota.  Requesting a larger quota may require user's
> >>  // explicit permission via UI prompting / infobar.
> >>
> >>  void requestQuota(
> >>  unsigned short storageType,
> >>  unsigned long long newQuotaInBytes,
> >>  optional StorageInfoQuotaCallback successCallback,
> >>  optional StorageInfoErrorCallback errorCallback);
> >> };
> >>
> >>
> >> This API works across all storage types except localStorage, as
> localStorage
> >> is unique.
> >>
> >> It spans IndexedDB, FileSystem, AppCache and for those that are carrying
> it,
> >> WebSQL.
> >>
> >>
> >> -Charles
> >>
> >>
> >
>
>

Re: [XHR2] Blobs, names and FormData

[Michael Nordman]

2011/8/23 Jonas Sicking 

> On Tue, Aug 23, 2011 at 12:53 AM, Anne van Kesteren 
> wrote:
> > On Tue, 23 Aug 2011 02:25:38 +0200, Jonas Sicking 
> wrote:
> >>
> >> Given that this seems to be the way people prefer to do it, I'm fine
> >> with doing that for Gecko too. But we definitely would need to get
> >> FormData.append changed so that it supports the use case.
> >
> > So the current line for Blob can be replaced with
> >
> >  append(DOMString name, Blob value, optional DOMString filename);
> >
> > right?
>
> Yes. Where if the filename is excluded is left out it uses the
> File.name of the Blob (if the Blob is also a File), or "blob" (if the
> Blob is not a File).
>
> That leaves the question what to do if the filename is specified but
> is the empty string. I would argue that we should honor the call and
> use the empty string as name when submitting.
>
> / Jonas
>

SGTM

Re: [XHR2] Blobs, names and FormData

[Michael Nordman]

A few of us on the chrome team just discussed this. We see this as a
deficiency in the FormData interface and intend to address it by providing a
variant of FormData.append(...) that allows the caller to provide a
'filename' when appending a blob. We don't intend to implement
BlobBuilder.getFile().

Anne, would you be willing to massage the XHR2 spec to allow for the
additional param in the case of a blob?

On Mon, Jul 18, 2011 at 3:00 PM, Jonas Sicking  wrote:

> On Mon, Jul 18, 2011 at 2:52 PM, Charles Pritchard 
> wrote:
> > On 7/18/2011 2:42 PM, Jonas Sicking wrote:
> >>>
> >>> >  If there is an API that replies on File to
> >>> >  work correctly we think we should fix it to work with Blob. For
> >>> > example, FileReader is
> >>> >  really BlobReader and works fine with Blobs. To me, getFile() should
> >>> > be unnecessary and
> >>> >  the best fix for FormData is to just modify append for this
> situation.
> >>
> >> While File and Blob are indeed very similar, they are also not the
> >> same thing. I think being able to create files is an easy addition for
> >> implementations, while it means not restricting getting Files from the
> >> users file system. So if you have code that expects to read a File
> >> from IndexedDB, the page can either populate that from the users
> >> filesystem, or by generating the data manually and using
> >> BlobBuilder.getFile.
> >
> > I'm very interested in this use case; I've been told that there may be
> > issues
> > with IndexedDB transaction constraints and FileReader/writer.
> >
> > Afaik, filewriter doesn't apply to blobs in this case. I don't know the
> > consequences of
> > callbacks to reader functions in the event loop, and whether that would
> end
> > up "blocking"
> > idb events. I'd imagine the blob is simply reset, and the stale reference
> > remains until it can
> > be cleaned up.
>
> Note that File objects are immutable just like Blobs are, so none of
> this applies here.
>
> The issues that you're talking about come up once we start talking
> about FileEntries, but that's an entirely different beast which are
> unrelated to this thread. No-one has suggested being able to create a
> FileEntry using BlobBuilder, nor that FileEntries are required to
> submit named content using FormData.
>
> Hope that helps.
>
> / Jonas
>
>

Re: [XHR2] Blobs, names and FormData

[Michael Nordman]

The problem is around naming the binary parts attached to
multi-part-form-encoded FormData. I think I'm in favor of the more  direct
solution to this problem, providing a FormData.append() variant that
optionally  allows the caller to specify a name. If no name is provided and
the blob is a File, the filename is used, otherwise a non-empty value
generated by  the client is used.

  formdata.append("elementName", blob, "picture.png");

BlobBuilder.getFile() as a solution to this particular problem is less
direct (and less obvious and less discoverable so less friendly). And it
raises questions like... Is the blob data really flattened out as a file on
the disk somewhere? If not, then getFile() is somewhat misleading. Is so, we
may have to create actual files when we otherwise wouldn't. Where is this
file created? When does it get deleted?

2011/7/18 Adrian Bateman 

> On Monday, July 11, 2011 12:46 PM, Charles Pritchard wrote:
> > Problem is too strong a statement. I am all for trivial changes, part of
> my
> > advocacy for getFile is from past experiences when blob was less
> supported;
> > getFile would have helped.
> >
> > FileReader has base64 encoding for binary data-- base64 encoding
> otherwise
> > only works on DOMString.
> >
> > I'd like to see both proposals implemented... Then I get everything!
> > On Jul 11, 2011, at 12:21 PM, Adrian Bateman 
> wrote:
> > > It requires more work for us. Our createObjectURL doesn't require that
> abstraction.
> > > The difference here is in the ECMAScript type. In contrast, modifying
> FormData
> > > append is a trivial change.
> > >
> > > What are the other APIs where this is a problem?
>
> Our view is to see everything here as a Blob. A File is a specialisation of
> Blob that
> happens to have a name and modified date. If there is an API that replies
> on File to
> work correctly we think we should fix it to work with Blob. For example,
> FileReader is
> really BlobReader and works fine with Blobs. To me, getFile() should be
> unnecessary and
> the best fix for FormData is to just modify append for this situation.


> Adrian.
>

Re: Offline Web Applications status

[Michael Nordman]

2011/4/7 Michael Nordman 

>
>
> 2011/4/7 louis-rémi BABE 
>
>> Thank you all for your valuable answers.
>>
>> There seems to be a pretty solid agreement on "ability to exclude the
>> master page form the cache".
>> Michael, you are suggesting using a different way of referring to the
>> manifest: 
>> Why not just let it be listed in the NETWORK section of the manifest?
>> It would make the means of including or excluding resources from the
>> cache more consistent.
>>
>
> I think the  tag syntax is a better API for the reasons mentioned in
> the whatwg list, i'll reiterate them here...
>
> > A few ideas on syntax...
> > a. 
> > b. If the main resource has a "no-store" header, don't add it to the
> > cache, but do associate the document with the cache.
> > c. A new manifest section to define a prefix matched namespace for these
> pages.
> Option (c) isn't precisely targeted at individual resources, authors
> would have to arrange their URL namespace more carefully to achieve
> the desired results. Option (a) and (b) can be applied specifically to
> individual pages offering greater control over. And option (a) is the
> more explicit the two. Authors need only edit the page to get the
> desired result instead of having to arrange for the appropiate http
> headers. Readers of the code (and view sourcers of the pages) can more
> easily determine how this page should utilize the appcache just by
> looking at the source of the page.
>
> My pick would be option (a), 
>
> The summary is that the  is cleaner and clearer
> imo. The NETWORK section doesn't actually exclude resources from being added
> to the cache. For example you can explicitly list a url in the CACHE
> section, and again in the NETWORK section... guess what... the resource is
> added to the cache and will be utilized.
>
>
Also, I'm looking for a syntax that adds this new capability in a
non-breaking way. I want to avoid changing the semantics of the any existing
API since it's already shipped.


>
>> >Allow cross-origin HTTPS resources to be included in manifest files.
>> >This much is actually done already in chromium impl as described on the
>> whatwg list.
>> Is it already available in a stable version of Chrome?
>>
>
> Chrome 11 should hit the stable channel in ~18 days. It's in the beta
> channel now.
>
>
>> I would also like to see this one fully implemented in Firefox asap.
>> I'll get in touch with our developers.
>>
>
> Fantastic! I'll open a bug in webkit to do the same since.
>
>
>> Micheal, have you got use-cases for the 4th and 5th feature request of
>> this thread [1]?
>>
>
> This feature is more architecture driven than use-case driven. Without such
> a feature, while in many cases there may be a way to produce a solution that
> functions 'offline',  the trouble is that it's a completely different
> solution than in the 'online' case. I'll follow up with you about these
> features in particular.
>
>
>
>> And could you please keep me updated with your implementation efforts
>> in Chromium?
>>
>
> I would be very happy to and likewise would be appreciated.
>
>
>>
>> Regards,
>>
>> Louis-Rémi
>>
>> [1]
>> http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1121.html
>>
>>
>

Re: Offline Web Applications status

[Michael Nordman]

2011/4/7 louis-rémi BABE 

> Thank you all for your valuable answers.
>
> There seems to be a pretty solid agreement on "ability to exclude the
> master page form the cache".
> Michael, you are suggesting using a different way of referring to the
> manifest: 
> Why not just let it be listed in the NETWORK section of the manifest?
> It would make the means of including or excluding resources from the
> cache more consistent.
>

I think the  tag syntax is a better API for the reasons mentioned in
the whatwg list, i'll reiterate them here...

> A few ideas on syntax...
> a. 
> b. If the main resource has a "no-store" header, don't add it to the
> cache, but do associate the document with the cache.
> c. A new manifest section to define a prefix matched namespace for these
pages.
Option (c) isn't precisely targeted at individual resources, authors
would have to arrange their URL namespace more carefully to achieve
the desired results. Option (a) and (b) can be applied specifically to
individual pages offering greater control over. And option (a) is the
more explicit the two. Authors need only edit the page to get the
desired result instead of having to arrange for the appropiate http
headers. Readers of the code (and view sourcers of the pages) can more
easily determine how this page should utilize the appcache just by
looking at the source of the page.

My pick would be option (a), 

The summary is that the  is cleaner and clearer
imo. The NETWORK section doesn't actually exclude resources from being added
to the cache. For example you can explicitly list a url in the CACHE
section, and again in the NETWORK section... guess what... the resource is
added to the cache and will be utilized.



>
> >Allow cross-origin HTTPS resources to be included in manifest files.
> >This much is actually done already in chromium impl as described on the
> whatwg list.
> Is it already available in a stable version of Chrome?
>

Chrome 11 should hit the stable channel in ~18 days. It's in the beta
channel now.


> I would also like to see this one fully implemented in Firefox asap.
> I'll get in touch with our developers.
>

Fantastic! I'll open a bug in webkit to do the same since.


> Micheal, have you got use-cases for the 4th and 5th feature request of
> this thread [1]?
>

This feature is more architecture driven than use-case driven. Without such
a feature, while in many cases there may be a way to produce a solution that
functions 'offline',  the trouble is that it's a completely different
solution than in the 'online' case. I'll follow up with you about these
features in particular.



> And could you please keep me updated with your implementation efforts
> in Chromium?
>

I would be very happy to and likewise would be appreciated.


>
> Regards,
>
> Louis-Rémi
>
> [1]
> http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1121.html
>
>

Re: How to standardize new Offline Web app features? [Was Re: Offline Web Applications status]

[Michael Nordman]

> How to standardize new Offline Web app features?

Something that can help with 'standardizing' a new feature is an
implementation. Maybe I can help on that one by building the list of
features mentioned earlier in this thread [1] into chrome. The first three
bullets are fairly easy to come by but the fourth bullet will take some
time, but once four is done, five should just fall out.

[1] http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1121.html
<http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1121.html>

On Fri, Apr 1, 2011 at 1:18 PM, Michael Nordman  wrote:

> Hi Art,
>
> Please don't assume I know how the w3c works. I'm not subscribed to the
> public-html list and honestly don't have  a good understanding of which list
> is for what. I consider the feature set provided in by the Application Cache
> to harmonize with other topics discussed on the public-webapps list, so it
> seemed like the natural place to discuss it. Actually Louis-Rémi started
> the thread to which I responded.
>
> Interesting question about moving the Offline Web Applications out of HTML5
> to a new home. I doesn't matter to me where this is spec'd or discussed so
> much, what matters is that progress is made as I think it's clear there is
> demand for "more" in this area. I have not been party to any discussions
> about relocating the Offline WebApps part of the HTML5 spec. Since you
> asked, I'm guessing you think that could help with making faster progress?
>
> -Michael
>
> On Fri, Apr 1, 2011 at 3:51 AM, Arthur Barstow wrote:
>
>> Michael, All,
>>
>> On Mar/31/2011 6:18 PM, ext Michael Nordman wrote:
>>
>>> I have in mind several extensions to the ApplicationCache that I think
>>> could address some of the additional desirements from the web developement
>>> community. I'll post them here because people seem to be more willing to
>>> have a discussion on the topic here than over in whatwg.
>>>
>>
>> From the process perspective, I think it is fine to discuss this feature
>> on public-webapps but since Offline Web applications is defined in the HTML5
>> spec, I am curious why you didn't use the public-html list.
>>
>> BTW, has there been any discussion (e.g. in the WHATWG or HTMLWG) about
>> moving the Offline Web application  out of the HTMLWG's HTML5 spec and into
>> a separate spec? I'm wondering if that could help facilitate the
>> standardization of new features like those you proposed in this thread [1].
>>
>> -Art Barstow
>>
>> [1]
>> http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1121.html
>>
>>
>>
>

Re: How to standardize new Offline Web app features? [Was Re: Offline Web Applications status]

[Michael Nordman]

Hi Art,

Please don't assume I know how the w3c works. I'm not subscribed to the
public-html list and honestly don't have  a good understanding of which list
is for what. I consider the feature set provided in by the Application Cache
to harmonize with other topics discussed on the public-webapps list, so it
seemed like the natural place to discuss it. Actually Louis-Rémi started the
thread to which I responded.

Interesting question about moving the Offline Web Applications out of HTML5
to a new home. I doesn't matter to me where this is spec'd or discussed so
much, what matters is that progress is made as I think it's clear there is
demand for "more" in this area. I have not been party to any discussions
about relocating the Offline WebApps part of the HTML5 spec. Since you
asked, I'm guessing you think that could help with making faster progress?

-Michael

On Fri, Apr 1, 2011 at 3:51 AM, Arthur Barstow wrote:

> Michael, All,
>
> On Mar/31/2011 6:18 PM, ext Michael Nordman wrote:
>
>> I have in mind several extensions to the ApplicationCache that I think
>> could address some of the additional desirements from the web developement
>> community. I'll post them here because people seem to be more willing to
>> have a discussion on the topic here than over in whatwg.
>>
>
> From the process perspective, I think it is fine to discuss this feature on
> public-webapps but since Offline Web applications is defined in the HTML5
> spec, I am curious why you didn't use the public-html list.
>
> BTW, has there been any discussion (e.g. in the WHATWG or HTMLWG) about
> moving the Offline Web application  out of the HTMLWG's HTML5 spec and into
> a separate spec? I'm wondering if that could help facilitate the
> standardization of new features like those you proposed in this thread [1].
>
> -Art Barstow
>
> [1]
> http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1121.html
>
>
>

Re: Offline Web Applications status

[Michael Nordman]

Hi again,

I have in mind several extensions to the ApplicationCache that I think could
address some of the additional desirements from the web developement
community. I'll post them here because people seem to be more willing to
have a discussion on the topic here than over in whatwg.

1. Allow cross-origin HTTPS resources to be included in manifest files. This
much is actually done already in chromium impl as described on the whatwg
list.

2. Allow a syntax to associate a page with an application cache, but does
not add that page to the cache. A common feature request also mentioned on
the whatwg list, but it's not getting any engagement from other browser
vendors or the spec writer (which is kind of frustrating). The premise is to
allow pages vended from a server to take advantage of the resources in an
application cache when loading subresources. A perfectly reasonable
request, .

3. Introduce a new manifest file section to INTERCEPT requests into a prefix
matched url namespace and satisfy them with a cached resource. The resulting
page would be free to interpret the location url and act accordingly based
on the path and query elements beyond the prefix matched url string. This
section would be similar to the FALLBACK section in that prefix matching is
involved, but different in that instead of being used only in the case of a
network/server error, the cached INTERCEPT resource would be used
immediately w/o first going to the server.
   INTERCEPT:
   urlprefix redirect newlocationurl
   urlprefix return cachedresourceurl

Here's where the INTERCEPT namespace could fit into the changes to the
network model.
if (url is EXPLICITLY_CACHED)  // exact match
  return cached_response;
if (url is in NETWORK namespace) // prefix match
  return network_response_as_usual;
if (url is in INTERCEPT namespace) // prefix match < this is the new
section 
  return handle_intercepted_request_accordingly
if (url is in FALLBACK namespace) // prefix match
  return
repsonse_as_usual_unless_fallback_conditions_are_met_by_that_response;
otherwise
  return synthesized_error_response;

4. Allow an INTERCEPT cached resources to be "executable". Instead of simply
returning the cached resource in response to the request, load it into a
background worker context (if not already loaded) and invoke a function in
that context to asynchronously compute response headers and body based on
the request headers (including cookie) and body. The background worker would
have access to various local storage facilities (fileSystem, indexed/sqlDBs)
as well as the ability to make network requests via XHR.
   INTERCEPT:
   urlprefix execute cachedexecutableresourceurl

5. Create a syntax to allow FALLBACK resources to be similarly executable in
a background worker context.

As an author of gears and chromium's appcache, I've been interested in
pursuing something along these lines for a while. Chrome is finally getting
built up to the point where this could be done, the file system and better
support for many worker threads are prerequisite.

With the outline above in mind, this is why earlier in the thread i
questioned the need for an explicit add(url) and remove(url) sort of api.
That capability can be composed out of the file system and an executable
intercept handler. And there is also window.createObjectUrl() and
FileEntry.toUrl() which could cover many use cases that are behind the
feature request for add() and remove().

-Michael


On Sat, Mar 26, 2011 at 8:32 PM, Jack Coulter  wrote:

> IndexedDB would be more suited to what you're doing Nathan, I've always
> seen
> ApplicationCache as something to only use on the core HTML/JS/CSS and
> perhaps
> small images, like icons (none of this would change often, and would
> generally
> be rather small) whereas IndexedDB sounds more like what you did with
> gears.
>
> Or, there's always the File System API, but I don't think it's widely
> implemented yet.
>
> Excerpts from Nathan Kitchen's message of Sun Mar 27 08:46:35 +1100 2011:
> > A couple of other app cache observations from a hobbyist who's played
> around
> > with Google's Gears...
> >
> > I built an offline web application based on Gears, with the intention to
> > migrate to something a bit more standardized as it became available. That
> > was a good two years ago now, but the existing and proposed
> implementations
> > still don't offer the capability that I can get from Gears.
> >
> > If you're in Chrome, point your browser at: http://my.offlinebible.com/.
> My
> > observations are going to be related to this app. I apologise if they're
> > quite specific, but they present a snapshot of some real-world issues
> > encountered while developing an offline app.
> >
> >- *App Cache*
> >The first task that the app undertakes is downloading a large amount
> of
> >offline data. This data includes the usual web page resources, plus a
> >massive amount of data destined for the Gears/WebS

Re: How many ways to save store app data?

[Michael Nordman]

Not sure if this is what you're looking for, but the
http://www.html5rocks.com/ site provides a developer facing view of these
things.

On Mon, Mar 28, 2011 at 6:21 AM, Arthur Barstow wrote:

> Hi All,
>
> Louis-rémi's thread [1] on AppCache led to discussions about other storage
> related APIs including DataCache, Google Gears, IDB and the File * APIs.
>
> Are there are any good resources that describe the various storage APIs
> (from the app developer's perspective) and compare their main properties
> such as: primary Use Cases addressed, strengths, weaknesses, implementation
> and deployment status, performance characterizations, security concerns,
> features planed for next version, etc.? The APIs of interest include: HTML5
> AppCache, DataCache, File * APIs, Google Gears, Indexed Database, Web
> Storage, and Web SQL Database.
>
> In the absence of such resource(s), would anyone be interested in leading,
> or contributing to, an informative document like this?
>
> WebApps does have a related wiki document that was created primarily to
> rationalize the start of the IDB spec vis-à-vis Web SQL Database. This
> document hasn't been touched in over a year and probably needs updating:


>   http://www.w3.org/2008/webapps/wiki/Database
>
> -Art Barstow
>
> [1]
> http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1043.html
>
>
>
>
>

Re: Offline Web Applications status

[Michael Nordman]

Hi Louis,

It's good to see some interest in the AppCache from the mozilla camp. My
take on one thing that "what went wrong" is that the community of browser
vendors stopped engaging on the topic, so I'm happy to see you posting here.
You pointed out two features that have been often requested.

- no way to exclude the referring document
- no means to add/remove ad-hoc resources

>From time to time, these requests have reached the whatwg list (where the
AppCache feature set is specified), but nothing much comes of them for lack
of engagement.

> Maybe you think that the problem has to do with the specification itself.

It is a static HTTP resource caching mechanism that is narrowly focused on
the "offline" use case. The narrow focus can make it difficult to use for
other use-cases (the struggle you mentioned some developers have with using
it to boost performance for online apps). And more significantly, a static
resource caching mechanism doesn't fully satisfy the "offline" use case.
Consider http://myapp/view/
?mode=printable.

> Do you think it *can* be used as an auxiliary cache mechanism...

Not really, for the same reason you mentioned. How about we fix that.
There's a thread on the whatwg about this topic specifically.
http://www.mail-archive.com/whatwg@lists.whatwg.org/msg24516.html

> Why isn't there any DOM API allowing a fine-grained control ...

The ability to add/remove ad-hoc resources was removed from the feature set
at an early date at least partly because the update logic would need to be
considerably more complicated to handle them. Also, since then other storage
apis have been specified that allow for the storage of large amounts of
ad-hoc data (FileSystem). I'm aware of apps under development that are using
the FileSystem for ad-hoc resources, but the ability to access/load the file
system data transparently via HTTP urls is still missing.

Given the other storage mechanisms that are available, I question whether
adding and removing ad-hoc resources from the appcache is needed. I'm
interested in seeing the AppCache evolve in the direction of closing the gap
between a requested url and the data stored in
IndexeDBs/FileSystem/WebSQLDatabases, being able to compute a response based
on the contents of those storage repositories. Others on the chrome team
have an interest in this as well.

> Maybe there is an evolution of the specification underway that I am not
aware of.

There was the DataCache draft, but that was less of an evolution than a
different thing altogether. Progress on that being dropped, but there were
some interesting ideas represented in that draft.
http://www.w3.org/TR/2009/WD-DataCache-20091029/

-Michael

2011/3/23 louis-rémi BABE 

> Hello Webapps working group,
>
> I'm an intern at Mozilla Developers Engagement team and I'm currently
> working on promoting Offline Web Applications.
> My first task is to understand what did go wrong with the App Cache
> mechanism...
>
> ## Maybe Web devs don't use App Cache because they don't understand
> what it is... ##
> The possibility of using Webapps offline has a great potential but its
> adoption by developers didn't reach our expectations. We asked Web
> developers some time ago what were their feelings regarding
> application cache  (see
> http://hacks.mozilla.org/2010/05/revitalizing-caching/ ) and it
> appeared that the name was misleading, as they expected it to be more
> of an auxiliary cache than an offline mechanism. You can find
> evidences of this confusion on StackOverflow, where some people
> struggle to use the application cache as a mean to boost performances
> of their Websites.
>
> ## Can you see other reasons? ##
> Before going back to developers or writing yet another App Cache
> documentation, I wanted to have *your* feelings about this mechanism.
> You might have a different impression about its adoption and be aware
> of successful real-world use-cases.
> You might have asked developers yourself and received a different feedback.
> Maybe you feel that Web advocates are not doing a good enough job at
> documenting this feature, producing demos and clarifying its nature.
> Maybe you think that the problem has to do with the specification itself.
> Maybe there is an evolution of the specification underway that I am
> not aware of.
>
> ## Two naive questions ##
> After reading a large amount of documentation, I have to admit that I
> am myself confused about app cache:
> Do you think it *can* be used as an auxiliary cache mechanism, and
> what would be the limitations? The main problem I see is that there is
> no way to white-list the referring document (e.g. index.html).
> Currently, I would advocate *against* using it as an auxiliary cache.
> Why isn't there any DOM API allowing a fine-grained control of the
> application cache?
> applicationCache.cache.add( URI );
> applicationCache.cache.remove( URI );
>
>
> Thank you in advance,
>
> Louis-Rémi Babé
>
>

Re: Updates to FileAPI

[Michael Nordman]

On Tue, Dec 21, 2010 at 11:31 AM, Arun Ranganathan  wrote:
> I have updated the editor's draft of the  File API (special winter solstice
> edition).
>
> In particular:
>
> On 12/20/10 8:32 PM, Michael Nordman wrote:
>>
>> On Mon, Dec 20, 2010 at 4:08 PM, Arun Ranganathan
>>  wrote:
>>>
>>> On 12/20/10 5:38 PM, Jian Li wrote:
>>>
>>> On Mon, Dec 20, 2010 at 2:10 PM, Ian Hickson  wrote:
>>>>
>>>> On Mon, 20 Dec 2010, Arun Ranganathan wrote:
>>>>>
>>>>> http://dev.w3.org/2006/webapi/FileAPI/
>>>>>
>>>>> Notably:
>>>>>
>>>>> 1. lastModifiedDate returns a Date object.
>>>
>>> I think it makes more sense to return a new Date object each time. We
>>> have
>>> the same issue with Metadata.modificationTime.
>>>
>
> There are more rigid conformance requirements around lastModifiedDate.
>
> http://dev.w3.org/2006/webapi/FileAPI/#dfn-lastModifiedDate
>
> (Ensure that your previous copy has been flushed).
>
>
>> I think we have the expectation that any published URLs, that have not
>> been explicitly revoked earlier, are jetisoned at some well defined
>> document cleanup time... what hixie pointed to looks like a great
>> place...
>>
>> http://www.whatwg.org/specs/web-apps/current-work/complete.html#unloading-document-cleanup-steps
>>
> I've bolstered the lifetime conformance language with this:
> http://dev.w3.org/2006/webapi/FileAPI/#lifeTime
>
> The only nagging doubt I have about this proposal is that the creation and
> revocation methods (static) are on window.URL, but the actual Blob object
> has an affiliated document, which upon cleanup flushes the URLs affiliated
> with it.
>
> We still need to address nits about event queue, abort, and sequence, but
> those will come in due course.


"When a user agent unloads a document, as part of the unloading
document cleanup steps, it MUST revoke a Blob URI which dereferences a
Blob object in that document. Implementations MUST ensure that a Blob
URI is revoked after URL.revokeObjectURL is called with that Blob URI
as an argument. Implementations MUST respond with a 404 Not Found if a
Blob URI is dereferenced after URL.revokeObjectURL is called on that
paticular Blob URI."

I don't know what "a Blob object in that document" means? That
language is a confusing to me.

The cleanup steps should revoke blob URIs that were created via a call
to the URL.createObjectURL from within that document.  Two different
documents (docA and docB) may create two different URLs (urlA and
urlB) for the same Blob. When docA is cleaned up, urlA should be
revoke but urlB should not.


> -- A*
>
>
>

Re: Updates to FileAPI

[Michael Nordman]

On Mon, Dec 20, 2010 at 4:08 PM, Arun Ranganathan  wrote:
> On 12/20/10 5:38 PM, Jian Li wrote:
>
> On Mon, Dec 20, 2010 at 2:10 PM, Ian Hickson  wrote:
>>
>> On Mon, 20 Dec 2010, Arun Ranganathan wrote:
>> >
>> > http://dev.w3.org/2006/webapi/FileAPI/
>> >
>> > Notably:
>> >
>> > 1. lastModifiedDate returns a Date object.
>>
>> You don't have a conformance requirement for returning a Date object. (The
>> only MUST is for the case of the UA not being able to return the
>> information.) I mention this because for attributes that return
>> objects, it's important to specify whether the same object is returned
>> each time or whether it's a new object that is created each time.
>> Presumably for a Date object you want to require a new object be created
>> each time.
>
> I think it makes more sense to return a new Date object each time. We have
> the same issue with Metadata.modificationTime.
>
> I agree with Ian and Jian Li that it makes sense to return a new Date
> object, and we should have conformance language about this.  I'll fix this
> very shortly.
>
>>
>>
>> > 2. We use the URL object and expose static methods on it for Blob URI
>> > creation and revocation.
>>
>> Looks good to me. FYI, I'm probably going to be extending this mechanism
>> for Streams in due course. I expect I'll bring this up again in due course
>> so we can work out how to make sure the specs don't step on each other.
>>
>> I'm a little concerned about the lifetime of these URLs potentially
>> exposing GC behaviour -- we've tried really hard not to expose GC
>> behaviour in the past, for good reason. Can't we jetison the URLs as part
>> of the unloading document cleanup steps?
>>
>>
>> http://www.whatwg.org/specs/web-apps/current-work/complete.html#unloading-document-cleanup-steps
>>
>> (Note that Window objects in some edge cases can survive their Document.)
>>
>
> Hmm... in the past on this issue, attempts to determine *when exactly* the
> URLs were jetisoned weren't successful (and hence we have an explicit
> revocation method).  There's a long-ish thread on this very topic on this
> listserv.
>
> I *suppose* we can harness this to document cleanup steps, but I'm
> interested to hear from others.

I think we have the expectation that any published URLs, that have not
been explicitly revoked earlier, are jetisoned at some well defined
document cleanup time... what hixie pointed to looks like a great
place...
http://www.whatwg.org/specs/web-apps/current-work/complete.html#unloading-document-cleanup-steps


>>
>> > Also, I've minuted Sam Weinig at TPAC saying he'd prefer us to roll back
>> > from using the sequence type WebIDL syntax to index getters.  Sam:
>> > are you still tightly wed to this?  WebIDL has undergone changes since
>> > last we spoke.  I'm copying what HTML5 is doing, and didn't want to be
>> > inconsistent in rolling this back.
>>
>> FWIW, IIRC the HTML spec is a bit out of sync when it comes to WebIDL.
>>
>
> I think that sequence is syntactically desirable, but I'm not sure
> present WebIDL accounts for *.index() behavior, which could be why Sam wants
> to roll back.
>
> -- A*
>

Re: revokeObjectURL behavior

[Michael Nordman]

On Thu, Nov 18, 2010 at 2:17 PM, Jonas Sicking  wrote:
> On Thu, Nov 18, 2010 at 2:04 PM, Michael Nordman  wrote:
>> On Tue, Nov 16, 2010 at 11:42 AM, Jonas Sicking  wrote:
>>> Hi All,
>>>
>>> We have at length discussed what revokeObjectURL should be called, and
>>> where it should live. However we haven't yet discussed how it should
>>> behave.
>>>
>>> In particular, there is one edge case that I'm concerned about. Consider:
>>>
>>> myurl = window1.URL.createObjectURL(myblob);
>>> window2.URL.revokeObjectURL(myurl);
>>>
>>> in this example window1 and window2 are separate, but same origin,
>>> windows (for example window2 could be an iframe inside window1).
>>>
>>> The question is, should the call to revokeObjectURL succeed, do
>>> nothing, or throw an exception. One important aspect here is that if
>>> the two windows are not same origin, and the code in window2 simply
>>> guesses a url to revoke, then we definitely don't want the revoke to
>>> succeed. While implementations should take steps to make URLs
>>> unguessable, it is good to have extra layers of security by not
>>> allowing different origins to unregister each others URLs.
>>>
>>> Another concern I have is that silently doing nothing is bad for APIs
>>> that are intended to free resources. It makes it very easy to create
>>> the situation where a author think they are revoking a URL, but in
>>> reality are not.
>>>
>>> I think for safety, I'm leaning towards saying that different
>>> same-origin windows can unregister each others URLs. But if
>>> revokeObjectURL is called with a string that is not a same-origin URL,
>>> it does nothing (other than possibly warning in error consoles if the
>>> UA so desires).
>>
>> Given the per-window semantics of these URLs, it seems a odd that a
>> URL coined in one window is revocable via another window.  Is there a
>> use-case in mind that would rely on being able to revoke thru any of
>> the origin's windows?
>>
>> The other obvious choice is that a window only knows how to revoke a
>> URL that it has explicitly coined which I think has clearer semantics.
>> These semantics, along with silent error handling, match what is
>> implemented in webcore right now (albeit with the methods bound to the
>> window object for now).
>>
>> Jian... you said... "This is what we currently assume."
>> Do you have changes in the works to alter the current semantics/impl
>> in webcore? If so, what's motivating that change?
>
> My thinking was that since this is a "free resources" API being as
> permissive as possible will result in the fewest leaks.
>
> Look at it this way. If a page on site A calls revokeObjectURL, it
> probably does so in order to free up the resources that that url is
> holding. Why would you want to ignore that call and still hold the
> resources just because the resource was originally allocated in
> another window on site A? It seems very likely to me that the result
> will be that those resources are leaked (until the user leaves the
> page). It seems much less likely that the author made the call in
> error and is planning on using the resource later.
>
> It's bad enough that we have to have a specific call to free
> resources. I think we should make it as easy as possible for authors
> to use the call, even if it's at the cost of additional implementation
> complexity.
>
> / Jonas

Got it. I can appreciate that.

Re: revokeObjectURL behavior

[Michael Nordman]

On Tue, Nov 16, 2010 at 11:42 AM, Jonas Sicking  wrote:
> Hi All,
>
> We have at length discussed what revokeObjectURL should be called, and
> where it should live. However we haven't yet discussed how it should
> behave.
>
> In particular, there is one edge case that I'm concerned about. Consider:
>
> myurl = window1.URL.createObjectURL(myblob);
> window2.URL.revokeObjectURL(myurl);
>
> in this example window1 and window2 are separate, but same origin,
> windows (for example window2 could be an iframe inside window1).
>
> The question is, should the call to revokeObjectURL succeed, do
> nothing, or throw an exception. One important aspect here is that if
> the two windows are not same origin, and the code in window2 simply
> guesses a url to revoke, then we definitely don't want the revoke to
> succeed. While implementations should take steps to make URLs
> unguessable, it is good to have extra layers of security by not
> allowing different origins to unregister each others URLs.
>
> Another concern I have is that silently doing nothing is bad for APIs
> that are intended to free resources. It makes it very easy to create
> the situation where a author think they are revoking a URL, but in
> reality are not.
>
> I think for safety, I'm leaning towards saying that different
> same-origin windows can unregister each others URLs. But if
> revokeObjectURL is called with a string that is not a same-origin URL,
> it does nothing (other than possibly warning in error consoles if the
> UA so desires).

Given the per-window semantics of these URLs, it seems a odd that a
URL coined in one window is revocable via another window.  Is there a
use-case in mind that would rely on being able to revoke thru any of
the origin's windows?

The other obvious choice is that a window only knows how to revoke a
URL that it has explicitly coined which I think has clearer semantics.
These semantics, along with silent error handling, match what is
implemented in webcore right now (albeit with the methods bound to the
window object for now).

Jian... you said... "This is what we currently assume."
Do you have changes in the works to alter the current semantics/impl
in webcore? If so, what's motivating that change?

>
> Let me know what you think.
>
> / Jonas
>
>

Re: Discussion of File API at TPAC in Lyon

[Michael Nordman]

On Tue, Nov 16, 2010 at 10:42 AM, Dmitry Titov  wrote:
> I have a clarifying question about how it would work:
>
> How, if at all, the lifetime of the generated urls will be defined in case
> of having those functions on URL interface object?
>
> If those methods are on a global object, the lifetime of the urls created
> would be gated by the lifetime of that global object, and in browser
> implementations that lifetime is usually defined well - and used for other
> things like lifetime of shared workers for example. There is certain times
> when the document/page/window are 'closed and detached', and while it varies
> in implementations and it is more complex then that, it provides a
> well-understood lifetime boundary.
>
> By having those methods on some static object, doesn't the lifetime becomes
> unclear? Do we grab 'current context of the call'?

The intent (as i understand it) was to not change the lifetime
semantics that had been devised when these methods were defined at the
global scope, just to put a 'namespace' around them (so to speak). The
goal being a little less clutter the global namespace.

>
> Dmitry
>
> On Tue, Nov 16, 2010 at 8:10 AM, Jonas Sicking  wrote:
>>
>> On Tuesday, November 16, 2010, Anne van Kesteren  wrote:
>> > On Tue, 16 Nov 2010 01:35:05 +0100, Jonas Sicking 
>> > wrote:
>> >
>> > Ok, here is what I'll propose as the final solution:
>> >
>> > FileAPI will define the following WebIDL:
>> >
>> > [Supplemental]
>> > interface URL {
>> >   static DOMString createObjectURL(in Blob blob);
>> >   static void revokeObjectURL(in DOMString url);
>> > };
>> >
>> > [...]
>> >
>> > Unless I hear otherwise from people, I'll assume that everyone is
>> > happy with this.
>> >
>> >
>> > This looks great. Will this make it into Firefox 4?
>>
>> Assuming we can get everyone to agree quickly enough, yes.
>>
>> / Jonas
>>
>
>

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Michael Nordman]

Personally, I don't think new response modes is the proverbial straw
that breaks the camel's back.

I don't see the problem with selecting the responseType up front
either. It doesn't feel like a new class of object is warranted just
to provide the response body different forms. As Jonas pointed out,
any errors that occur when a new response mode is utilized (that makes
responseText unavailable to legacy library code) would be immediately
apparent to the developer attempting to use the new response mode and
easy enough to resolve.

Some arguments against new object types...

* If we do introduce a new object type that is nearly identical to the
existing object type,  that may introduce more compatibilities issues
than modest extensions to the existing object. Think script that is
designed to work with XHR being re-purposed for use with the new
nearly identical object, but not functioning properly because the new
object type is only "nearly" identical.

* Its that much more request handling bloatware in script to operate
with the different object types.

* Its that much more binding bloatware.




On Wed, Nov 10, 2010 at 2:42 AM, Anne van Kesteren  wrote:
> On Tue, 09 Nov 2010 21:03:22 +0100, Jonas Sicking  wrote:
>>
>> I still don't see the problem with the responseType. There's very
>> little difference in both BinaryHttpRequest and responseType is opt-in
>> mechanisms.
>
> I am not a big fan of a new object either. It would require two new objects,
> for one. It seems everyone keeps forgetting about the anonymous variant for
> convenience.
>
> During the meeting we discussed allowing responseType to be set just until
> after the headers were received so content negotiation could in theory still
> work. Presumably it would start throwing from the state LOADING onwards
> otherwise you get into trouble with disk-backed APIs (i.e. Blob) versus
> synchronous APIs (i.e. Document). This would not work with a new object that
> requires you to make a choice before things are being retrieved.
>
> It would be nice if we had something better than responseType though. It is
> very confusing given responseXML and responseText imo.
>
>
>> The only difference is that .responseType allows a existing object
>> (which a library could be holding a reference to) could be "mutated"
>> since it now behaves differently. It seems to me that whenever this is
>> done in the "wrong" way code should consistently fail, and so should
>> be easy for developers to deal with.
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: [DataCache] Status and plans for Programmable HTTP Caching and Serving?

[Michael Nordman]

There are two distinct parts to the DataCache draft.
http://www.w3.org/TR/2009/WD-DataCache-20091029/
1. The DataCache part, programmatically adding resources to the cache, the
DataCache part.
2. The LocalServer part, having application script determine/produce the
response to a request, handling 'POST's in the absence of a server.

To satisfy the use cases motivating the first item (bulk storage with
URL addressability), we've (chrome) been pushing on WebFileSystem and blob
URL. The latter part has been left languishing.

I'm all for incremental progress on the "local server" like missing parts,
specifically starting with "what to do about query arguments", in the
context of the appcache. We (chrome) have specific requests to improve
things in that area. I think we could make faster progress if we handle some
of these requests as incremental additions to the HTML5 AppCache spec
instead of resurrecting DataCache draft.

At least, that's my $0.02.


On Fri, Nov 5, 2010 at 6:26 AM, Arthur Barstow wrote:

>  Hi All,
>
> During WebApps' November 1 gathering at the TPAC, we discussed [Mins] the
> WG's "Programmable HTTP Caching and Serving" spec, last updated by Nikunj in
> January 2010 [DataCache].
>
> During this discussion, I agreed to the following related actions:
>
> ACTION-599 "Ask Nikunj to report the status and plans of Programmable Cache
> to public-webapps"
> http://www.w3.org/2008/webapps/track/actions/599
>
> ACTION-600 "Ask Oracle about their level of interest in Programmable Cache"
> http://www.w3.org/2008/webapps/track/actions/600
>
> ACTION-601 "Ask public-webapps about creating Use Cases and requirements of
> Program App Caches versus HTML5 App Cache"
> http://www.w3.org/2008/webapps/track/actions/601
>
> ACTION-602 "Contact julian reschke and Mark Nottingham about Data Cache"
> http://www.w3.org/2008/webapps/track/actions/602
>
> My short summary of this discussion is: it isn't clear if there is
> sufficient interest to continue this spec and if there is interest, what are
> the specific use cases it will address (i.e. use cases not addressed by
> HTML5 AppCache), and who is willing to commit to drive the spec forward.
>
> -Art Barstow
>
> [ tracker tags: ACTION-599, ACTION-600, ACTION-601, ACTION-602 ]
>
> [Mins] http://www.w3.org/2010/11/01-webapps-minutes.html#item05
> [DataCache] http://dev.w3.org/2006/webapi/DataCache/
>
>
>  Original Message   Subject: ACTION-600: Ask Oracle about
> their level of interest in Programmable Cache (Web Applications Working
> Group)  Date: Mon, 1 Nov 2010 11:40:04 +0100  From: ext Web Applications
> Working Group Issue Tracker   
> Reply-To:
> Web Applications Working Group WG 
>   To:
> Barstow Art (Nokia-CIC/Boston) 
>
> ACTION-600: Ask Oracle about their level of interest in Programmable Cache 
> (Web Applications Working Group)
> http://www.w3.org/2008/webapps/track/actions/600
>
> On: Arthur Barstow
> Due: 2010-11-08
>
> If you do not want to be notified on new action items for this group, please 
> update your settings 
> at:http://www.w3.org/2008/webapps/track/users/7672#settings
>
>

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Michael Nordman]

On Wed, Oct 27, 2010 at 12:34 PM, Boris Zbarsky  wrote:

> On 10/27/10 3:23 PM, Michael Nordman wrote:
>
>> Darin mentioned it earlier in this thread, having XHR return the raw
>> data and providing other interfaces to interpret/decode the raw data
>> into strings/xmlDocs. I like that decomposition.
>>
>
> If we were designing from scratch, yes.
>
> Or is the proposal that we introduce the up-front modal switch and provide
> such APIs to allow consumers who want bytes-or-string to say they want bytes
> but then manually create the string later?
>
>
There wasn't a concrete proposal, just saying the division of labor between
data retrieval and data processing sounded good to me. I would be in favor
of concrete proposals in that direction and I view the responseArrayBuffer
attribute as a step in that direction.


> Note, by the way, that for XML and HTML types, one is required (per current
> XHR spec) to instantiate an XML or HTML parser, respectively, to produce the
> responseText (because the data can declare its own encoding in things like
>  tags).  I'm not sure whether that complicates the other interfaces
> being proposed I guess they could do that under the hood.  But they
> point is they'd need to know the MIME type, not just the data.
>

So an XMLParser would need more than just the raw data, ok.


>
> -Boris
>

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Michael Nordman]

Darin mentioned it earlier in this thread, having XHR return the raw data
and providing other interfaces to interpret/decode the raw data into
strings/xmlDocs. I like that decomposition.

> Because I'm skeptical that nested libraries trying to access the data both
ways will be an issue.

I'm not sure that is an issue either. To the extent it is an issue,
libraries could be updated to be sensitive to whether asArrayBuffer/asBlob
is set.

On Wed, Oct 27, 2010 at 11:56 AM, Geoffrey Garen  wrote:

> Hi Boris.
>
> > As long as the same object provides both views and the views are mutually
> exclusive, it's really easy for the dispatcher to change what they ask for
> and subtly break other consumers he doesn't even know about.
>
> To the contrary, I think eliminating subtlety is the main benefit of
> explicitly selecting a response type and throwing exception on incorrect
> access.
>
> Quietly providing multiple copies of data creates a subtle memory use
> problem.
>
> Explicitly throwing an exception at the site that tries to access the data
> in the wrong way obviously and immediately points a finger at the problem.
>
> The potential downside to an exception is not subtlety -- it's limited
> functionality: clients are no longer allowed to ask XHR to be more than one
> kind of data at once. The question is, can library authors make their code
> compatible with this limitation?
>
> I don't think it's possible to answer that question until someone fleshes
> out your assumption that libraries routinely intercept XHRs in transit and
> read their data. Which libraries do this, when, and why?
>
> > This make it impossible to decide whether to look at the text or the
> bytes based on the content-type of the response (unless you allow setting
> the attribute in some early-enough onStateChange callback _and_ libraries
> expose XHRs in that early a state to consumers); having that ability seems
> like a desirable feature.
>
> I'm not convinced that clients loading resources from their own domains
> somehow won't know the types of those resources.
>
> But, if we believe this is a real concern, it seems fine to me to allow
> responseType to be set some time after open, or even set implicitly by the
> first access to a certain view of the response. That way, clients wanting to
> delay the responseType decision can, while clients wanting maximum clarity
> and control can set responseType early.
>
> Thanks,
> Geoff

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Michael Nordman]

I'm in favor of adding the explicit asArrayBuffer attribute. Simple and
straight forward for both web developers and browser developers.


On Tue, Oct 26, 2010 at 3:39 PM, Chris Rogers  wrote:

>
>
> On Mon, Oct 25, 2010 at 3:33 PM, Boris Zbarsky  wrote:
>
>> On 10/25/10 6:21 PM, Chris Rogers wrote:
>>
>>>  People are concerned that it would require keeping two copies of the
>>> data around (raw bytes, and unicode text version) since it's unknown
>>> up-front whether "responseText", or "responseArrayBuffer" will be
>>> accessed.
>>>
>>
>> Note that Gecko does exactly that, and we've seen no problems with it...
>>  It's very rare to have really large XHR bodies, for what it's worth.
>
>
> Alexey and James Robinson from Google have experienced noticeable
> performance-related issues with keeping two copies.  Mobile devices are
> going to suffer especially from these problems.  Also, Chris Marrin points
> out that the payloads are likely to become MUCH larger when dealing with
> binary data.  It's simply wasteful to decode to text when it was clearly not
> intended to be used as text.  Performance isn't something we can just
> ignore.
>
> 2. Darin Fisher has suggested this approach: Add an "asArrayBuffer"
>>> attribute (similar to "asBlob") which *must* be set before sending the
>>> request if there will be a subsequent access of the
>>> "responseArrayBuffer" attribute.
>>>
>>
>> This make it impossible to decide whether to look at the text or the bytes
>> based on the content-type of the response (unless you allow setting the
>> attribute in some early-enough onStateChange callback _and_ libraries expose
>> XHRs in that early a state to consumers); having that ability seems like a
>> desirable feature.
>
>
> We already have an "asBlob" attribute which has the same issue.  Why is it
> so unreasonable to also have "asArrayBuffer".  If there's a different method
> we devise to also determine based on content-type when we receive the
> header, then that's fine too, but it doesn't seem to preclude the idea of
> having "asArrayBuffer".
>
>
>
>>
>>
>>  3. Get rid of the "asBlob" attribute and add a new "responseType"
>>> attribute which could be:
>>> "Text" <--- this is the default
>>> "XML"
>>> "Bytes"
>>> ... other types yet to be defined
>>>
>>
>> I'm not sure I follow this proposal.
>
>
> This was just an alternative to having "asBlob" and "asArrayBuffer"
> attributes, where instead there's a "responseType" attribute which can be
> set up front to one of several types.
>
>
>
>>
>>
>>  I can accept any of the proposed solutions and would like to hear what
>>> others think about these or other approaches.
>>>
>>
>> How about:
>>
>> 4)  Make things easy to use for authors; that means supporting
>> responseText and responseArrayBuffer, with access to both on the same XHR
>> object without weird restrictions and without clairvoyance required on the
>> part of the author.  If UAs feel that they don't want to keep both copies of
>> the data in memory on a permanent basis, they can optimize this by dropping
>> the responseText (on a timer or via a low-memory detection mechanism, or in
>> various other ways) and regenerating it from the raw bytes as needed.
>>
>> ?
>>
>
> I think we simply disagree about the performance issues involved.  Adding
> an "asArrayBuffer" or similar which can be set up-front does not hurt the
> current API which is used currently to receive text.  The normal API usage
> continues to behave as it does and would not break any existing libraries.
>  Any future libraries which will use the "responseArrayBuffer" attribute can
> easily be coded sensibly with "asArrayBuffer".  I think that in the vast
> majority of use cases, it *will* be known up-front what the type of the data
> is, so clairvoyance should not be an issue.  If we decide to also devise an
> approach which can make use of content-type information in the response,
> then that's fine too.
>
> Chris
>
>
>
>

Re: createBlobURL

[Michael Nordman]

Chrome currently has this as...
DOMString createBlobURL(in Blob blob);
void revokeBlobURL(in DOMString blobURL);
... no prefix... on both Window and WorkerContext.

An important aspect of placing these interfaces on the Window and
WorkerContext was to scope the lifetime of the manufactured url to the
respective window or worker object.  I thought adam's url api was all about
parsing and building valid urls, so i'm not sure these methods are such a
good fit for that interface.

Renaming to createObjectURL/revokeObjectURL seems like the most
straightforward way to generify the existing methods.

On Mon, Oct 18, 2010 at 11:50 AM, Arun Ranganathan  wrote:

>  On 10/18/10 12:16 PM, Anne van Kesteren wrote:
>
>> On Mon, 18 Oct 2010 18:09:24 +0200, Jonas Sicking 
>> wrote:
>>
>>> I would be ok with moz prefixing, though it would be nice to try to
>>> find a solution pretty quickly if all this is is a discussion about
>>> finding an appropriate name for this function.
>>>
>>
>> Well, if people agree with me that the appropriate place would be the URL
>> object -- assuming for now we are going to get that -- it would be more
>> involved than just renaming.
>>
>
> Moving these features to the URL object is more complex, but may be
> desirable.  I'm not sure I can shoe horn that big an adjustment in time for
> the CfC (not least of all because there isn't a draft of the URL API I can
> refer to.  I suppose I could specify the object that Adam has started with
> in the File API draft, though).  Maybe more discussion is warranted at a
> venue like the TPAC.
>
> However, the revocation capability is pretty essential.  So this is
> something we'd have to add to the URL API.  And, the URL API has to be
> robust enough for both Stream and Blob use cases.
>
> Since Anne's suggestion is an important but late-breaking one, I'd propose
> tabling a CfC till the editor's draft accurately reflects consensus on what
> to do with URL generation and revocation for Blobs.  Maybe one thing we can
> do with existing implementation trains is:
>
> 1. Use vendor prefixing for create* and revoke* according to some rules [1]
> (since underscores are rarely used in DOM APIs, we'd have
> window.moz_createBlobURL() or chrome_createBlobURL( ) ), but I'd really like
> to hear from the Chromium folks.  Darin?  Others?
>
> 2. Adding revoke* to Adam's URL *or* create more versatile APIs that are
> better named within window* that account for Stream *and* Blob.
>
> A benefit *and* a drawback of reusing the URL object is that this would
> make Blob URLs and HTTP URLs within the same object.  It makes sense to
> revoke a Blob URL, but much less sense to revoke another URL.  The schemes
> are different, but they *do* behave pretty much similarly.
>
> I'd like a bit more implementer feedback, especially since doing 2. would
> be a pretty late breaking change.  I'm in favor of the best solution, but
> hate late breaking changes.  Chrome folks?
>
> -- A*
> [1]
> https://wiki.mozilla.org/Tantek-Mozilla-projects#DOM_API_vendor_prefixing
>
>

Re: [XHR2] ArrayBuffer integration

[Michael Nordman]

A couple of us have been looking at webkit's XHR impl and realized that to
support performant access to the response via responseArrayBuffer and
responseText would cause us to keep two copies of the data around, a raw
data buffer and the second decoded text buffer. Considering the overwhelming
number of responseText only use cases, it would be nice to not incur extra
costs to support a new binary accessor which would be rarely used compared
to the text accessor. So some way of making these mutually exclusive in the
API.

Should we have an asArrayBuffer attribute (or similar) to tell XHR up front
how the caller wishes to access the response?

Or maybe specify the accessors such that when you use responseText you can
not later access responseArrayBuffer and vice versa?

Are there any use cases where you'd need it both ways?

-Michael


On Fri, Sep 24, 2010 at 5:41 PM, Kenneth Russell  wrote:

> On Fri, Sep 24, 2010 at 5:36 PM, Jian Li  wrote:
> > I plan to add ArrayBuffer support to BlobBuilder and FileReader. Chris,
> it
> > is good that you would pick up the work for XHR. We can talk about how
> we're
> > going to add ArrayBufferView to read ArrayBuffer.
>
> All of the Typed Array view types (Uint8Array, Float32Array, etc.)
> except for Float64Array are already implemented in WebKit. The major
> missing one for file and network I/O is DataView.
>
> -Ken
>
> > Jian
> >
> > On Fri, Sep 24, 2010 at 5:23 PM, Kenneth Russell  wrote:
> >>
> >> On Thu, Sep 23, 2010 at 2:42 AM, Anne van Kesteren 
> >> wrote:
> >> > On Wed, 08 Sep 2010 19:55:33 +0200, Kenneth Russell 
> >> > wrote:
> >> >>
> >> >> Mozilla's experimental name is "mozResponseArrayBuffer", so perhaps
> to
> >> >> avoid collisions the spec could call it responseArrayBuffer.
> >> >
> >> > While I do not think there would be collision (at least not in
> >> > ECMAScript,
> >> > which is what we are designing for) naming it responseArrayBuffer is
> >> > fine
> >> > with me. And also now done that way in the draft. Still need to get a
> >> > saner
> >> > reference to the ArrayBuffer specification than
> >> >
> >> >
> https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html
> >> > though. :-)
> >> >
> >> > http://dev.w3.org/2006/webapi/XMLHttpRequest-2/
> >>
> >> Thanks, this is great and very exciting. This motivates implementing
> >> the proposed DataView interface (
> >>
> >>
> https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html#6
> >> ), which will make it easier to read multi-byte values with specified
> >> endianness out of an ArrayBuffer. For WebKit I've filed
> >> https://bugs.webkit.org/show_bug.cgi?id=46541 .
> >>
> >> -Ken
> >>
> >> > (You can also do send(ArrayBuffer) obviously. I personally think
> >> > supporting
> >> > this for both BlobBuilder and send() makes sense. That way Blob/File
> >> > etc.
> >> > work too.)
> >> >
> >> >
> >> > --
> >> > Anne van Kesteren
> >> > http://annevankesteren.nl/
> >> >
> >>
> >
> >
>
>

Re: Blob/File naming

[Michael Nordman]

On Tue, Sep 7, 2010 at 12:12 PM, Adrian Bateman wrote:

> On Tuesday, September 07, 2010 11:46 AM, Chris Prince wrote:
> > >> 1. Most people that I talk to dislike the name Blob, much less having
> > >> it spread to things like BlobReader.
> >
> > I could maybe understand this if "blob" were a new term we were
> > inventing.  But it's not.  It's a well-known computer science concept.
> >  It seems worse to try and coin a totally new name for "opaque chunk
> > of data".
>
>
> > FWIW, "most people" hating the name blob seems like a stretch, as it
> > has not been my experience.  But maybe we run in different circles.
>
> This was addressed to me although quoting Jonas.
>
> One of the problems I've experienced is that in general the well-known
> computer
> science concept doesn't have a URL that can be used to stream data into
> another
> object. It's feasible to use the Blob interface in circumstances where the
> bits
> of the "blob" aren't manifested until they are actually used. Some concepts
> from
> the Media Capture API seem to be heading in this direction.
>

We've reformed the api that provides a url such that it's no longer an
attribute of the Blob instance. Instead there's a method of the window' that
binds the contents of the Blob to a url that is valid for the lifetime of
the window object or until the url is revoked via another window method.


>
> That said, I don't have a problem with the name Blob. :o)
>
>
Me  neither.


> Cheers,
>
> Adrian.
>
>

Re: XHR.responseBlob and BlobWriter [nee FileWriter]

[Michael Nordman]

On Tue, Aug 17, 2010 at 3:37 PM, Anne van Kesteren  wrote:

> On Wed, 18 Aug 2010 00:24:56 +0200, Michael Nordman 
> wrote:
>
>> Blob attribute responseBlob;
>> // Returns a blob the contains the response body.
>> // Only valid to access when asBlob is true and when the request is in
>> // a terminal state. Throws INVALID_STATE_ERR if accessed at an
>> // invalid time.
>>
>
> I suppose when asBlob is true and the state is not DONE it should simply
> return null like responseXML does for consistency. But when asBlob is false
> it should throw.
>

Sounds good (here and below) ... thnx for the clarification.


>
> See for responseXML:
>
> http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#the-responsexml-attribute
>
>
>
>  Currently for network errors (i.e. not "errors" such as HTTP 410)
>>> everything will return their default value. Is there a good reason to do
>>> this differently for responseBlob?
>>>
>>
>> Ok... so is the "default value" for responseBlob an empty blob when asBlob
>> is true and the attribute is accessed while the XHR object is in a terminal
>> error or aborted state? Just looking to clarify whether the accessor should
>> throw the invalid state error or return normally in these terminal states.
>>
>
> If we go with null above it should be null here too.
>
>
> Returning null is probably also better if we decide we want to expose the
> Blob as streaming entity at some point. I.e. while the state is LOADING.
>
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: XHR.responseBlob and BlobWriter [nee FileWriter]

[Michael Nordman]

Here's the text I've put in the webkit (and chrome) bug reports for this
feature...
https://bugs.webkit.org/show_bug.cgi?id=44133
http://code.google.com/p/chromium/issues/detail?id=52486

Implement an XHR.responseBlob accessor to get an opaque reference to
the response data as a blob.

Two new additions for the XHR interface in support of this. This is
not solidified yet, still pending some discussion on the public lists.

boolean attribute asBlob;
// Prepares the XHR to make the response available as a blob object.
// Defaults to false, must be set after calling open() and
// prior to calling send(). Gets reset upon subsequent calls to open().
// Throws INVALID_STATE_ERR if set at an invalid time. Maybe read at
// anytime without exception.

Blob attribute responseBlob;
// Returns a blob the contains the response body.
// Only valid to access when asBlob is true and when the request is in
// a terminal state. Throws INVALID_STATE_ERR if accessed at an
// invalid time.

When asBlob is true, the other response accessors (responseText,
resonseXML, responseBody) throw INVALID_STATE_ERR if accessed.

We're making this "modal" for the benefit of the browser vendors, to
make it easier for them to know how to handle the response data as it
comes in. With a priori knowlege that the data need never be made
available thru the responseText attribute, life is a little easier for
them/us.

Also see http://code.google.com/p/chromium/issues/detail?id=52486


On Tue, Aug 17, 2010 at 3:13 PM, Anne van Kesteren  wrote:

> On Tue, 17 Aug 2010 21:14:03 +0200, Michael Nordman 
> wrote:
>
>> On Mon, Aug 16, 2010 at 11:34 PM, Anne van Kesteren > >wrote:
>>
>>> Yeah I have. This design looks rather good. We should also define what
>>>
>>> happens to responseText and responseXML in this scenario. I suppose
>>> throwing
>>> INVALID_STATE_ERR makes the most sense to make it clear they are not
>>> available. Nothing else should be affected right?
>>>
>>
>> INVALID_STATE_ERR works. If responseText throws when asBlob is true, then
>> resposneBlob should throw when asBlob is false. Another plausible  option
>> is to return NULL or UNDEFINED from these calls in those cases. Either way
>> works, personally I like the unambiguous INVALID_STATE_ERR.
>>
>
> Good point about responseBlob. Lets go for the exception.
>
>
>
>  The responseBlob attribute should be available for access when the 'done'
>> events are raised, so withing the event handler, the caller should be able
>> to access that attribute and get final results. Here's what i'm referring to
>> as the 'done' events...
>> - onerror  (responseBlob yields whatever partial results there were if
>> any)
>>
>
> Currently for network errors (i.e. not "errors" such as HTTP 410)
> everything will return their default value. Is there a good reason to do
> this differently for responseBlob?


Ok... so is the "default value" for responseBlob an empty blob when asBlob
is true and the attribute is accessed while the XHR object is in a terminal
error or aborted state? Just looking to clarify whether the accessor should
 throw the invalid state error or return normally in these terminal states.


>
>
>  - onload
>> - onloadend
>> - readystate changing to DONE
>> The only one I wonder about is onabort, what is the behavior of accessing
>> reponseBlob in that terminating condition?
>>
>
> Same as for onerror at the moment.
>
>
>
>  I think nothing else changes, incremental ready state changed events and
>> progress events should still fire as usual.
>>
>
> Yes.
>
>
> (I'm going to wait at least a week or so with updating the draft to include
> this (and other) feature(s). XMLHttpRequest and XMLHttpRequest Level 2 are
> created from the same source document and I would like to give priority to
> integrating the proposed changes for XMLHttpRequest (Level 1) for which I
> hope to have WG consensus declared next week.)
>
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: XHR.responseBlob and BlobWriter [nee FileWriter]

[Michael Nordman]

On Mon, Aug 16, 2010 at 11:34 PM, Anne van Kesteren wrote:

> On Tue, 17 Aug 2010 04:05:52 +0200, Michael Nordman 
> wrote:
>
>> I'm interested in adding support for XHR.responseBlob to webkit and
>> chrome. It sounds like we have a general consensus to add two new members to
>> the
>> XmlHttpRequest interface:
>>
>>  attribute boolean asBlob;  // at least for the sake of discussion,
>> calling
>> it that for now
>>  readonly attribute Blob responseBlob;
>>
>> Another name option could be 'downloadAsBlob'. Personally, I like 'asBlob'
>> for its brevity and its naming similarity to  'withCredentials'.
>>
>> Looks like the semantics of when its valid to call setRequestHeaders also
>> applies setting the asBlob attribute; after open() but before send() has
>> been called. An attempt to set the value at some inappropriate time should
>> throw an INVALID_STATE exception. Also the asBlob attribute value should
>> probably be reset to 'false' upon return from open(), again similar to how
>> the request headers are cleared out upon return from open().
>>
>> Anne, have you been following this discussion, wdyt?
>>
>
> Yeah I have. This design looks rather good. We should also define what
> happens to responseText and responseXML in this scenario. I suppose throwing
> INVALID_STATE_ERR makes the most sense to make it clear they are not
> available. Nothing else should be affected right?
>

INVALID_STATE_ERR works. If responseText throws when asBlob is true, then
resposneBlob should throw when asBlob is false. Another plausible  option is
to return NULL or UNDEFINED from these calls in those cases. Either way
works, personally I like the unambiguous INVALID_STATE_ERR.

The responseBlob attribute should be available for access when the 'done'
events are raised, so withing the event handler, the caller should be able
to access that attribute and get final results. Here's what i'm referring to
as the 'done' events...
- onerror  (responseBlob yields whatever partial results there were if any)
- onload
- onloadend
- readystate changing to DONE
The only one I wonder about is onabort, what is the behavior of accessing
reponseBlob in that terminating condition?

I think nothing else changes, incremental ready state changed events and
progress events should still fire as usual.


>
>
> (I think you also found a bug in that the "credentials flag" and "request
> timeout" are not reset when open() is invoked.)
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: XHR.responseBlob and BlobWriter [nee FileWriter]

[Michael Nordman]

Hi All,

I'm interested in adding support for XHR.responseBlob to webkit and chrome.
It sounds like we have a general consensus to add two new members to the
XmlHttpRequest interface:

  attribute boolean asBlob;  // at least for the sake of discussion, calling
it that for now
  readonly attribute Blob responseBlob;

Another name option could be 'downloadAsBlob'. Personally, I like 'asBlob'
for its brevity and its naming similarity to  'withCredentials'.

Looks like the semantics of when its valid to call setRequestHeaders also
applies setting the asBlob attribute; after open() but before send() has
been called. An attempt to set the value at some inappropriate time should
throw an INVALID_STATE exception. Also the asBlob attribute value should
probably be reset to 'false' upon return from open(), again similar to how
the request headers are cleared out upon return from open().

Anne, have you been following this discussion, wdyt?

Michael

On Tue, Jun 29, 2010 at 4:31 PM, Jonas Sicking  wrote:

> On Tue, Jun 29, 2010 at 4:24 PM, Eric Uhrhane  wrote:
> > On Tue, Jun 29, 2010 at 11:28 AM, Jonas Sicking 
> wrote:
> >> On Mon, Jun 28, 2010 at 6:58 PM, Eric Uhrhane  wrote:
> >>> The discussion at [1] got tangled up with the debate of .URL vs. .url,
> >>> so I'm starting a new thread to pick back up the original topic: how
> >>> do we save binary data from XMLHttpRequest?  Here's my proposal [built
> >>> mainly from the good ideas other folks posted in the original thread].
> >>>
> >>> Use case 1: the developer wants to download a Blob of data, use it for
> >>> a while [e.g. slicing out sub-Blobs and displaying them as images],
> >>> then have it go away automatically.
> >>> Use case 2: the developer wants to download a Blob of data, saving it
> >>> in a location of the user's choice outside the sandbox.
> >>> Use case 3: the developer wants to download a Blob of data, save it in
> >>> the sandboxed FileSystem, and access it again later.
> >>>
> >>> XHR will have a responseBlob property.
> >>> In order to signal the XHR that it should spool to disk and supply
> >>> responseBlob, a flag must be set before send() is called.  Call this
> >>> wantBlob for now, although a better name would be appreciated.
> >>> If wantBlob is set, responseBlob will be valid when the XHR completes,
> >>> and responseText and responseXML will be null.
> >>> If wantBlob is not set, responseBlob will be null, and the XHR will
> >>> function as it does now.
> >>>
> >>> When wantBlob is set, on all progress events before the XHR is
> >>> complete, responseBlob will be null.  As of completion, it will return
> >>> a Blob containing the full contents of the download.  [We could later
> >>> add incremental Blobs in progress events, but let's keep this simple
> >>> to start with.]
> >>>
> >>> This satisfies use case 1 as-is.
> >>> With the BlobWriter spec [2], as currently written [assuming we agree
> >>> on how to get our hands on a BlobWriter], it takes care of use case 2.
> >>> With the FileSystem spec [3], as currently written, it takes care of
> use case 3.
> >>>
> >>> I think this takes care of the major use cases, doesn't force anyone
> >>> to implement FileSystem to solve the cases that don't really require
> >>> it, removes any need for synchronous IO, and is pretty simple for
> >>> developers to understand and use.  What do you think?
> >>
> >> Sounds great to me! Also note that this will allow
> >>
> >> Use case 3': the developer wants to download a Blob of data, save it in
> >> IndexedDB, and access it again later.
> >
> > I don't see Blob mentioned in the structured clone algorithm, although
> > File is there.  I doubt it would be much additional work to support all
> Blobs.
>
> Indeed, it should be trivial. I think Blob didn't exist yet at the
> time when the structured clone algorithm was last updated.
>
> / Jonas
>

Re: Lifetime of Blob URL

[Michael Nordman]

On Mon, Aug 2, 2010 at 1:39 PM, Jonas Sicking  wrote:

> On Fri, Jul 30, 2010 at 12:01 PM, Michael Nordman 
> wrote:
> >
> >
> > On Thu, Jul 29, 2010 at 4:33 PM, Jonas Sicking  wrote:
> >>
> >> Sorry about the slow response. I'm currently at blackhat, so my
> >> internet connectivity is somewhat... unreliable, so generally having
> >> to try to stay off the webs :)
> >>
> >> On Tue, Jul 27, 2010 at 1:16 PM, Dmitry Titov 
> wrote:
> >> > Thanks Jonas,
> >> > Just to clarify some details we had while discussing this, could you
> >> > confirm
> >> > if this matches with your thinking (or not):
> >> > 1. If blob was created in window1, blob.url was queried, then passed
> (as
> >> > JS
> >> > object) to window2, and window1 was closed - then the url gets
> >> > invalidated
> >> > when window1 is closed, but immediately re-validated if window2
> queries
> >> > blob.url. The url string is going to be the same, only there will be a
> >> > time
> >> > interval between closing window1 and querying blob.url in window2,
> >> > during
> >> > which loading from the url returns 404.
> >>
> >> Actually, it might make sense to make blob.url, when queried by
> >> window2, return a different string. This makes things somewhat more
> >> consistent as to when a URL is working an when not.
> >
> > Now suppose window2 queries the .url attribute before window1 is closed?
> I
> > think most people would expect the same value as returned in window1
> (yes?).
> > Having the same or different value depending on whether the attribute was
> > queried before or after another window was closed seems confusing. I
> think
> > having the .url remain consistent from frame to frame/window to window
> could
> > help with debugging.
>
> The idea would be that we *always* return different urls depending on
> which window queries a url. This gives the most consistent behavior in
> that every url given is always limited to the lifetime of the current
> window. No matter what windows around it does.
>

If that's the idea, then I would vote for a non-instance method somewhere to
provide the context specific URL. Having a simple attribute accessor return
different values depending on which context its being accessed in is very
unusual behavior.

Can't say that its "ideal", but window.getBlobUrl(blob) and
window.revokeBlobUrl(...) would be an improvement.


> > Without fully understanding the gap between blob and .url life time, some
> > folks are going to be mystified by when/why a url value stops working, or
> > why the .url value is sometimes different than it was before. There are
> some
> > pretty hidden side affect of accessing that attribute in a particular
> frame.
> > These subtle oddities with the .url attribute are in part
> > what originally motivated the proposal to make it more explicit.
> > We're trying to make blob.url easy and natural feeling, but with too many
> > caveats, will it be?
> > I guess that's a long winded vote for resurrecting the same url value
> used
> > in window1 in the particular case Dmitry raised.
>
> I totally agree that this is not an ideal solution. However as far as
> I can see there are no ideal solutions. The basic problem is that we
> are using a string to reference a resource, and there is no way to let
> a string value keep a resource alive.
>
> Usually resource management is done by holding a reference to that
> resource. Once there no longer are references to the resource the
> resource can be deleted without anyone noticing. Ideal would be if you
> could set:
>
> myImageElement.srcFile = myFile;
>
> However that would force us to add API to every single API that deals
> with urls, thus making that option too non-ideal. Instead we have
> chosen to have the ability to extract a url-string from the file,
> leaving us with other non-ideal behavior in the case when a url is
> transferred between windows.
>
> / Jonas
>

Re: Lifetime of Blob URL

[Michael Nordman]

On Thu, Jul 29, 2010 at 4:33 PM, Jonas Sicking  wrote:

> Sorry about the slow response. I'm currently at blackhat, so my
> internet connectivity is somewhat... unreliable, so generally having
> to try to stay off the webs :)
>
> On Tue, Jul 27, 2010 at 1:16 PM, Dmitry Titov  wrote:
> > Thanks Jonas,
> > Just to clarify some details we had while discussing this, could you
> confirm
> > if this matches with your thinking (or not):
> > 1. If blob was created in window1, blob.url was queried, then passed (as
> JS
> > object) to window2, and window1 was closed - then the url gets
> invalidated
> > when window1 is closed, but immediately re-validated if window2 queries
> > blob.url. The url string is going to be the same, only there will be a
> time
> > interval between closing window1 and querying blob.url in window2, during
> > which loading from the url returns 404.
>
> Actually, it might make sense to make blob.url, when queried by
> window2, return a different string. This makes things somewhat more
> consistent as to when a URL is working an when not.
>

Now suppose window2 queries the .url attribute before window1 is closed? I
think most people would expect the same value as returned in window1 (yes?).
Having the same or different value depending on whether the attribute was
queried before or after another window was closed seems confusing. I think
having the .url remain consistent from frame to frame/window to window could
help with debugging.

Without fully understanding the gap between blob and .url life time, some
folks are going to be mystified by when/why a url value stops working, or
why the .url value is sometimes different than it was before. There are some
pretty hidden side affect of accessing that attribute in a particular frame.
These subtle oddities with the .url attribute are in part
what originally motivated the proposal to make it more explicit.

We're trying to make blob.url easy and natural feeling, but with too many
caveats, will it be?

I guess that's a long winded vote for resurrecting the same url value used
in window1 in the particular case Dmitry raised.



>
> I.e. you're less likely to end up covering up a bug due to a URL
> coming back to life because another page started using a blob whose
> URL you were previously handed.
>
> It's a somewhat unlikely scenario so I don't feel very strongly either way.
>
> > 2. If blob is sent to a Worker via worker.postMessage(blob), the
> 'structured
> > clone' mechanism is used, so on the other side of postMessage a new blob
> > object is created, backed by the same data, but having its own unique
> > blob.url string (and separate lifetime).
>
> Yes.
>
> / Jonas
>
> > On Mon, Jul 26, 2010 at 2:12 PM, Jonas Sicking  wrote:
> >>
> >> On Tue, Jul 13, 2010 at 7:37 AM, David Levin  wrote:
> >> > On Tue, Jul 13, 2010 at 6:50 AM, Adrian Bateman <
> adria...@microsoft.com>
> >> > wrote:
> >> >>
> >> >> On Monday, July 12, 2010 2:31 PM, Darin Fisher wrote:
> >> >> > On Mon, Jul 12, 2010 at 9:59 AM, David Levin 
> >> >> > wrote:
> >> >> > On Mon, Jul 12, 2010 at 9:54 AM, Adrian Bateman
> >> >> > 
> >> >> > wrote:
> >> >> > I read point #5 to be only about surviving the start of a
> navigation.
> >> >> > As
> >> >> > a
> >> >> > web developer, how can I tell when a load has started for an ?
> >> >> > Isn't
> >> >> > this similarly indeterminate.
> >> >> >
> >> >> > As soon as img.src is set.
> >> >> >
> >> >> > "the spec could mention that the resource pointed by blob URL
> should
> >> >> > be
> >> >> > loaded successfully as long as the blob URL is valid at the time
> when
> >> >> > the
> >> >> > resource is starting to load."
> >> >> >
> >> >> > Should apply to xhr (after send is called), img, and navigation.
> >> >> >
> >> >> > Right, it seems reasonable to say that ownership of the resource
> >> >> > referenced
> >> >> > by a Blob can be shared by a XHR, Image, or navigation once it is
> >> >> > told
> >> >> > to
> >> >> > start loading the resource.
> >> >> >
> >> >> > -Darin
> >> >>
> >> >> It sounds like you are saying the following is guaranteed to work:
> >> >>
> >> >> img.src = blob.url;
> >> >> window.revokeBlobUrl(blob);
> >> >> return;
> >> >>
> >> >> If that is the case then the user agent is already making the
> >> >> guarantees
> >> >> I was talking about and so I still think having the lifetime mapped
> to
> >> >> the
> >> >> blob
> >> >> not the document is better. This means that in the general case I
> don't
> >> >> have
> >> >> to worry about lifetime management.
> >> >
> >> > Mapping lifetime to the blob exposes when the blob gets garbage
> >> > collected
> >> > which is a very indeterminate point in time (and is very browser
> version
> >> > dependent -- it will set you up for compatibility issues when you
> update
> >> > your javascript engine -- and there are also the cross browser issues
> of
> >> > course).
> >> > Specifically, a blob could go "out of scope" (to use your earlier
> >> > phrase)
> >> > and then one could do im

Re: [File API] Recent Updates To Specification + Co-Editor

[Michael Nordman]

On Thu, Jul 1, 2010 at 11:47 AM, Dmitry Titov  wrote:

> +1 to
>
> window.createBlobUrl(blob) and
> window.revokeBlobUrl(url)
>

+1 too, presumably these would also be available workers, particularly
shared workers.


> , they make lifetime of a blob url (which is different from lifetime of the
> Blob JS object) way more clear to me, since it attaches it explicitly to a
> window, and to a specific window in that. Multi-window web apps (like GMail
> or IM) have multiple windows that cross-script each other, so it's very easy
> to change the code a little and the 'spawning Document", as defined in the
> spec now, changes, since it's implicitly captured. Explicit is good.
>
>
By factoring the .url property out, the Blob object itself acts much more
like a String primitive in that it's an immutable container for data that
can be passed freely throughout a graph of connected browsing contexts, and
potentially serialized and sent via PostMessage. Overall simpler to grok w/o
the 'spawning context' concept.


> It also indeed makes it possible for a good-behaving app to prevent a
> memory leak in long-lived pages and potentially shared workers, since it
> gives an explicit 'revoke' method.
>
> Nice idea, lets see if it can be added to the spec.
>
> Dmitry
>
>
> On Wed, Jun 30, 2010 at 5:38 PM, Jian Li  wrote:
>
>> Thanks for the update. We've some more questions regarding the blob URL.
>>
>> 1. The spec does not describe how blob and blob URL will work in the
>> worker and shared worker scenarios. I think we should allow
>> WorkerGlobalScope to be the binding context for the blob URL, like Document.
>> In addition, we should define how a blob object can be passed to the worker
>> via structured cloning. A new blob object should be expected to be created
>> and it points to the same underlying data.
>>
>> 2. The current spec says that the lifetime of the blob URL is bound to the
>> lifetime of the spawning context. What happens if we try to access the blob
>> url from multiple contexts? Say, we
>> call "parent.blob.url", the lifetime of the url is bound to the parent
>> context, not the current context, per the spec. This sounds a little bit
>> unnatural. Could we explicitly provide the context while creating the blob
>> URL, like "window.createBlobUrl(blob)"?
>>
>> 3. Since the lifetime of the blob URL is bound to a context, the blob URL
>> (the underlying blob data) will get disposed only when the context dies.
>> When we have long-live pages or shared workers, we could have "leaked" blob
>> URLs that result in unclaimed blob storages. It will be nice if we can add
>> the capability to revoke the blob URL pragmatically,
>> like "window.revokeBlobUrl(url)",
>>
>> 4. It will be good if the spec could say more about the lifetime of the
>> blob object and the blob URL since they're kind of orthogonal: the blob
>> object will still be functional as long as it is not GC-ed even if the
>> associated context dies.
>>
>> 5. The spec does not describe explicitly about the transient cases, like
>> "location.href = blob.url". Probably the spec could mention that the
>> resource pointed by blob URL should be loaded successfully as long as the
>> blob URL is valid at the time when the resource is starting to load.
>>
>>
>> On Mon, Jun 28, 2010 at 2:20 PM, Arun Ranganathan wrote:
>>
>>> Greetings WebApps WG,
>>>
>>> I have made edits to the File API specification [1].  There are a few
>>> things of note that I'd like to call the WG's attention to.
>>>
>>> 1. There is a name change in effect.  FileReader has been re-named
>>> BlobReader, upon request from Chrome team folks[2][3].  The name
>>> "BlobReader" won't win awards in a beauty pageant, but it tersely describes
>>> an object to read Blobs (which could originate from the underlying file
>>> system *or* be generated *within* a Web App).  My present understanding is
>>> that FileWriter will also undergo a name change.  Naming is really hard.
>>>  Firefox already ships with FileReader, but I see the point of having an
>>> object named for what it does, which in this case is certainly more than
>>> file reading from the underlying file system.  I also abhor bike shedding,
>>> especially over naming, but this is something that's exposed to the authors.
>>>  I have not renamed FileError or FileException.  In the case of errors and
>>> exceptions, I think *most* scenarios will occur as a result of issues with
>>> the underlying file system.  These names should remain.
>>>
>>> 2. I've updated the URL scheme for Blobs using an ABNF that calls for an
>>> "opaque string" which is a term I define in the specification.  There was
>>> much discussion about this aspect of the File API specification, and I think
>>> the existing scheme does allow for user agents to tack on origin information
>>> in the URL (this is not something the spec. says you should do).  The actual
>>> choice of opaque string is left to implementations, though the specification
>>> suggests UUID in its canonical for

Re: BlobWriter simplification/split

[Michael Nordman]

On Wed, Jun 30, 2010 at 10:29 AM, Eric Uhrhane  wrote:

> On Wed, Jun 30, 2010 at 10:18 AM, Jonas Sicking  wrote:
> > On Tue, Jun 29, 2010 at 5:17 PM, Eric Uhrhane  wrote:
> >> Following up on discussions mainly at [1] and use cases at [2], I'd
> >> like to propose splitting the BlobWriter [née FileWriter] class, with
> >> an eye to solving some UI problems and simplifying implementation.
> >>
> >> When saving a Blob to a location outside the FileSystem API sandbox,
> >> we want to prompt the user exactly once, and want to be able to
> >> indicate that a write is taking place, e.g. by throbbing the download
> >> indicator.  We want the throbbing to go away as soon as the write is
> >> done, and we don't want the app to have continued privileges to write
> >> outside the sandbox.  [That's been debated, but I think it's beyond
> >> the scope of what we're working on so far, so let's leave that case
> >> for later expansion.]
> >>
> >> When writing inside the sandbox, we probably don't need the throbber
> >> at all, and we definitely don't want to prompt the user on each write.
> >>  Leaving aside the question of how one obtains a BlobWriter without
> >> using the sandbox and when exactly prompts happen [I'll open another
> >> thread for that], I think that:
> >>
> >> *  We don't want to have the same API call cause prompts to pop up on
> >> some instances of BlobWriter and not others.
> >> *  We don't want to have the same API call be reusable on some
> >> instances of BlobWriter and not others.
> >>
> >> I propose the following split:
> >>
> >> Define a parent class, SimpleBlobWriter, that's used for one-shot Blob
> >> export.  This is what you use when you don't have the FileSystem API,
> >> when the user selects the save location.  It contains a writeFile()
> >> method, which writes a Blob to a file in a single operation.  It is an
> >> error to call writeFile if readyState is not INIT, so SimpleBlobWriter
> >> is single-use.  Its other members are abort(), readyState, the ready
> >> state constants, error, length, and the progress event handlers, as
> >> defined in the current spec.
> >>
> >> Derive from SimpleBlobWriter the class BlobWriter, which adds
> >> position, truncate(), seek(), and write(), as defined in the current
> >> spec.  This is what the FileSystem API's createWriter() call would
> >> return.
> >>
> >> This lets you have different APIs for different behaviors, and removes
> >> fields from SimpleBlobWriter that aren't actually useful without the
> >> FileSystem API.
> >>
> >> How does that sound?
> >
> > Sounds great!
> >
> >> [Better names for SimpleBlobWriter and
> >> BlobWriter would be quite welcome, BTW.]
> >
> > May I propose FileWriter in place of BlobWriter? ;-)
> > You are actually always writing to files, so it would make a lot of sense
> IMO.
>
> We renamed BlobReader based on the perspective that it took data from
> a Blob and read it into memory.  Likewise BlobWriter takes data from a
> Blob and writes it to a file.  I think the symmetry makes sense.
> Calling it FileWriter also works, but then you're naming by
> destination instead of source, so I don't think it complements
> BlobReader as well.
>
> > An alternative would be to rename SimpleBlobWriter to FileSaver and
> > have the two be completely separate interfaces. It doesn't seem like
> > the inheritance is adding much anyway?
>
> I do like *Saver, though, since it clearly just saves an entire Blob.
> How about BlobSaver [with method save() instead of writeFile] and
> BlobWriter?
>
> The inheritance gets you length, abort(), error, the state constants,
> and all the progress events, which may have identical handlers in many
> cases.  I think that's probably enough to be worthwhile.  Anyone else
> want to chime in?
>

*Saver.save() is very good naming, short and sweet and tersely accurate.

As for the (*) noun. Whatever noun is chosen, it should probably be used for
the entire family (*Saver, *Reader, *Writer) for consistency. Also I don't
think its safe to assume that a *Saver interface will only ever be used to
dump a blob to a file (Jonas mentioned the possibility of dumping a blob
into an IndexedDB). These things make me lean towards Blob as the noun. Of
course... Y (bikeshed) MMV.

Re: XHR.responseBlob and BlobWriter [nee FileWriter]

[Michael Nordman]

On Mon, Jun 28, 2010 at 6:58 PM, Eric Uhrhane  wrote:

> The discussion at [1] got tangled up with the debate of .URL vs. .url,
> so I'm starting a new thread to pick back up the original topic: how
> do we save binary data from XMLHttpRequest?  Here's my proposal [built
> mainly from the good ideas other folks posted in the original thread].
>
> Use case 1: the developer wants to download a Blob of data, use it for
> a while [e.g. slicing out sub-Blobs and displaying them as images],
> then have it go away automatically.
> Use case 2: the developer wants to download a Blob of data, saving it
> in a location of the user's choice outside the sandbox.
> Use case 3: the developer wants to download a Blob of data, save it in
> the sandboxed FileSystem, and access it again later.
>
> XHR will have a responseBlob property.
> In order to signal the XHR that it should spool to disk and supply
> responseBlob, a flag must be set before send() is called.  Call this
> wantBlob for now, although a better name would be appreciated.
> If wantBlob is set, responseBlob will be valid when the XHR completes,
> and responseText and responseXML will be null.
> If wantBlob is not set, responseBlob will be null, and the XHR will
> function as it does now.
>
> When wantBlob is set, on all progress events before the XHR is
> complete, responseBlob will be null.  As of completion, it will return
> a Blob containing the full contents of the download.  [We could later
> add incremental Blobs in progress events, but let's keep this simple
> to start with.]
>
> This satisfies use case 1 as-is.
> With the BlobWriter spec [2], as currently written [assuming we agree
> on how to get our hands on a BlobWriter], it takes care of use case 2.
> With the FileSystem spec [3], as currently written, it takes care of use
> case 3.
>
> I think this takes care of the major use cases, doesn't force anyone
> to implement FileSystem to solve the cases that don't really require
> it, removes any need for synchronous IO, and is pretty simple for
> developers to understand and use.  What do you think?
>

SGTM


>
>   Eric
>
> [1]
> http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0313.html
> [2] http://dev.w3.org/2009/dap/file-system/file-writer.html
> [3] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
>

Re: Updates to File API

[Michael Nordman]

Another advantage is that...

blobdata://
http_responsible_party.org:80/3699b4a0-e43e-4cec-b87b-82b6f83dd752

... makes it clear to the end user who the responsible party is when these
urls are visible in the user interface. (location bar, tooltips, etc).

On Fri, Jun 11, 2010 at 11:11 AM, Jonas Sicking  wrote:

> On Fri, Jun 11, 2010 at 9:09 AM, Adrian Bateman 
> wrote:
> > On Wednesday, June 02, 2010 5:35 PM, Jonas Sicking wrote:
> >> On Wed, Jun 2, 2010 at 5:26 PM, Arun Ranganathan 
> wrote:
> >> > On 6/2/10 5:06 PM, Jian Li wrote:
> >> >> In addition,
> >> >> we're thinking it will probably be a good practice to encode the
> security
> >> >> origin in the blob URL scheme, like blobdata:
> >> >> http://example.com/33c6401f-8779-4ea2-9a9b-1b725d6cd50b. This will
> make
> >> >> doing the security origin check easier when a page tries to access
> the
> >> >> blob
> >> >> url that is created in another process, under multi-process
> architecture.
> >> >
> >> > This is a good suggestion.  I particularly like the idea of encoding
> the
> >> > origin as part of the scheme.
> >>
> >> Though we want to avoid introducing the concept of nested schemes to
> >> the web. While mozilla already uses nested schemes (jar:http://...
> >> and  view-source:http://...) I know others, in particular Apple, have
> >> expressed a dislike for this in the past. And with good reason, it's
> >> not easy to implement and has been a source of numerous security bugs.
> >> That said, it's certainly possible.
> >
> > It's not clear to me the benefit of encoding the origin into the URL. Do
> we expect script to parse out the origin and use it? Even in a multi-process
> architecture there's presumably some central store of issued URLs which will
> need to store origin information as well as other things?
>
> The one advantage I can see is that putting the scheme into the URL
> allows the *implementation* to deduce the origin by simply looking at
> the URL-scheme. This avoids having to do a (potentially cross-process)
> lookup to get the origin.
>
> This could be useful for APIs which have to synchronously determine
> the origin of a given URL in order to throw an exception on an
> attempted cross-origin access. For example an XMLHttpRequest Level 1
> implementation needs to synchronously determine if it should make a
> call to .open(...) throw or not based on the origin of the passed in
> URL.
>
> However I'm not sure if this is a problem in practice or not. It's
> entierly possible that the web platform is littered with situations
> where you need to do synchronous communication with whichever thread
> the networking code runs on.
>
> Firefox is still in the process of going multi-process, so I'll defer
> to other browsers with more experience in this area.
>
> / Jonas
>
>

Re: [IndexedDB] What happens when the version changes?

[Michael Nordman]

This sounds very similar to the strategy used for WebSQLDatabase.

On Tue, May 18, 2010 at 1:54 AM, Jonas Sicking  wrote:

> On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher 
> wrote:
> > On 5/13/2010 7:51 AM, Nikunj Mehta wrote:
> >>
> >> If you search archives you will find a discussion on versioning and that
> >> we gave up on doing version management inside the browser and instead
> leave
> >> it to applications to do their own versioning and upgrades.
> >
> > Right, I'm not saying we should manage it, but I want to make sure we
> don't
> > end up making it very easy for apps to break themselves.  For example:
> > 1) Site A has two different tabs (T1 and T2) open that were loaded such
> that
> > one got a script (T1) with a newer indexedDB version than the other (T2).
> > 2) T1 upgrades the database in a way that T2 now gets a constraint
> violation
> > on every operation (or some other error due to the database changing).
> >
> > This could easily happen any time a site changes the version number on
> their
> > database.  As the spec is written right now, there is no way for a site
> to
> > know when the version changes without reopening the database since the
> > version property is a sync getter, implementations would have to load it
> on
> > open and cache it, or come up with some way to update all open databases
> > (not so fun).
>
> I think what we should do is to make it so that a database connection
> is version specific. When you open the database connection (A) the
> implementation remembers what version the database had when it was
> opened. If another database connection (B) changes the version of the
> database, then any requests made to connection A will fail with a
> WRONG_VERSION_ERR error.
>
> The implementation must of course wait for any currently executing
> transactions in any database connection to finish before changing the
> version.
>
> Further the success-callback should likely receive a transaction that
> locks the whole database in order to allow the success callback to
> actually upgrade the database to the new version format. Not until
> this transaction finishes and is committed should new connections be
> allowed to be established. These new connections would see the new
> database version.
>
> / Jonas
>
>

Re: [IndexedDB] Granting storage quotas

[Michael Nordman]

Sounds like we agree on there being a distinction between two levels of
persistence with one being more permanent than the other. Great, provided
we have that agreement we can craft interfaces that allow callers to make
the distinction!

On Thu, Apr 29, 2010 at 10:57 AM, Jonas Sicking  wrote:

> On Thu, Apr 29, 2010 at 2:43 AM, Robin Berjon  wrote:
> > On Apr 29, 2010, at 01:35 , Jonas Sicking wrote:
> >> On Wed, Apr 28, 2010 at 4:17 PM, Eric Uhrhane  wrote:
> >>> On Wed, Apr 28, 2010 at 3:42 PM, Jonas Sicking 
> wrote:
>  * We'd like to expire data in IndexDB after some time. This will
>  likely be based on heuristics, such as haven't visited the site for an
>  extended period of time, though possibly keep the data a bit longer if
>  it was put in the database during offline mode and thus likely
>  contains data from the user. So in other words, we'd like to prevent
>  data staying on the users system indefinitely for a site that the user
>  no longer uses.
> >>>
> >>> Just to clarify: you're looking to expire storage that's marked as
> >>> persistent [or something like that] or that's marked as temporary, or
> >>> you're not planning to distinguish strongly between types of storage?
> >>
> >> We're definitely looking at expiring both. Though possibly with
> >> different levels of aggressiveness.
> >
> > I find this somewhat puzzling. As a user, if you've shown me a UI telling
> me you're asking for permanent storage to store the data I'm using in
> conjunction with a given app, you'd better never, ever so much as muse about
> expiring it, no matter how meekly. If you do, and I lose all those captioned
> kittens I'd been working on but put on the back burner, you can be sure I'll
> show up at your place to have a discussion about aggressive expiration. What
> am I missing?
>
> I think we were operating under the assumption that we're going to
> avoid involving the user until neccesary. So for example letting the
> site store a few MB of data without the user getting involved, and
> only once enough storage is wanted by the site, ask the user if this
> is ok.
>
> Asking on the first byte written does change things I agree. We'd have
> to look into that.
>
> / Jonas
>
>

Re: [IndexedDB] Granting storage quotas

[Michael Nordman]

This thinking resonates with what we've been thinking too (I think).

On Wed, Apr 28, 2010 at 3:42 PM, Jonas Sicking  wrote:

> We had some discussions about this at mozilla yesterday. I think the
> summary is something like this:
>
> * We'd like to expire data in IndexDB after some time. This will
> likely be based on heuristics, such as haven't visited the site for an
> extended period of time, though possibly keep the data a bit longer if
> it was put in the database during offline mode and thus likely
> contains data from the user. So in other words, we'd like to prevent
> data staying on the users system indefinitely for a site that the user
> no longer uses.
> * We'll expose UI to allow the user to remove data at any point.
> Similar to cookie UIs.
> * We'll expose APIs to extension developers to allow extensions to
> manage data lifetime as well, similar to cookie APIs.
> * We support the idea of temporary object stores used to for example
> implement cross joins. One solution is that these as never inserted
> into a database, but are only useable if you have a reference to a
> store object. Once the storage object is no longer used and is GCed,
> the store is nuked.
> * There was some support for non-permanent-but-possibly-long-lasting,
> useful for for example caching data that is also available on the
> server. The main concern here was that sites will always just opt for
> the most permanent storage option. So we need to ensure that there is
> incentive not to do this.


We have in mind that the incentives for developers to not always utilize the
most permanent storage option are...
1) Non-permanent storage is accessible w/o explicit user opt-in, so less
annoying UI.
2) Less of a burden to 'manage' the data if the user-agent will clean it up
for you.


> We definitely realize that automatically expiring data stored in
> IndexDB needs to be done with much care. Whatever the spec says,
> developers will likely think of the storage area as basically
> permanent. So we'll likely need to experiment with the expiring a good
> deal.
>
> Ultimately it probably doesn't make sense to have MUST requirements
> around this as this can't be tested anyway. I.e. there is no way to
> test that data put in the storage won't be expired in half a year. It
> might make sense to have some informal language in the spec so that
> authors have some idea of how this works, but for now we have no
> language to recommend since we'd like to get implementation
> experience.
>
> Again, this is similar to how cookies work, where we (and I believe
> other browsers), have played around with different expiration policies
> throughout the years.
>
> / Jonas
>
> On Wed, Apr 28, 2010 at 2:54 PM, Dumitru Daniliuc 
> wrote:
> > shawn, did you have a chance to give this some thought? how would mozilla
> > like to handle cases like the ones jeremy and robin mentioned? how would
> you
> > like to manage quotas?
> >
> > thanks,
> > dumi
> >
> >
> > On Fri, Apr 23, 2010 at 11:08 AM, Shawn Wilsher 
> wrote:
> >>
> >>  On 4/23/2010 7:39 AM, Nikunj Mehta wrote:
> >>>
> >>> Could we create an additional optional parameter for an open request
> with
> >>> the type of permanence required? Or is it not a good idea?
> >>
> >> I haven't talked to anyone at Mozilla that thinks that having permanent
> >> and non-permanent-but-possibly-long-lasting data to be a good idea.
>  There
> >> does seem to be support for a session-only version of indexedDB.
> >>
> >> Cheers,
> >>
> >> Shawn
> >>
> >
> >
>

Re: XMLHttpRequest.responseBlob

[Michael Nordman]

On Wed, Apr 28, 2010 at 11:21 AM, Jonas Sicking  wrote:

> Ugh, sent this originally to just Darin. Resending to the list.
>
> On Wed, Apr 28, 2010 at 10:11 AM, Darin Fisher  wrote:
> > On Tue, Apr 27, 2010 at 2:04 PM, Jonas Sicking  wrote:
> >>
> >> On Tue, Apr 27, 2010 at 1:59 PM, Darin Fisher 
> wrote:
> >> > On Tue, Apr 27, 2010 at 1:33 PM, Jonas Sicking 
> wrote:
> >> >>
> >> >> On Tue, Apr 27, 2010 at 1:26 PM, Darin Fisher 
> >> >> wrote:
> >> >> >> It would be nice to be able to allow streaming such that every
> time
> >> >> >> a
> >> >> >> progress event is fired only the newly downloaded data is
> available.
> >> >> >> The UA is then free to throw away that data once the event is done
> >> >> >> firing. This would be useful in the cases when the page is able to
> >> >> >> do
> >> >> >> incremental parsing of the resulting document.
> >> >> >>
> >> >> >> If we add a 'load mode' flag on XMLHttpRequest, which can't be
> >> >> >> modified after send() is called, then streaming to a Blob could
> >> >> >> simply
> >> >> >> be another enum value for such a flag.
> >> >> >>
> >> >> >> There is still the problem of how the actual blob works. I.e. does
> >> >> >> .responseBlob return a new blob every time more data is returned?
> Or
> >> >> >> should the same Blob be constantly modifying? If modifying, what
> >> >> >> happens to any in-progress reads when the file is modified? Or do
> >> >> >> you
> >> >> >> just make the Blob available once the whole resource has been
> >> >> >> downloaded?
> >> >> >>
> >> >> >
> >> >> >
> >> >> > This is why I suggested using FileWriter.  FileWriter already has
> to
> >> >> > deal with
> >> >> > most of the problems you mentioned above,
> >> >>
> >> >> Actually, as far as I can tell FileWriter is write-only so it doesn't
> >> >> deal with any of the problems above.
> >> >
> >> > When you use createWriter, you are creating a FileWriter to an
> existing
> >> > File.
> >> > The user could attempt to create a FileReader to the very same File
> >> > while
> >> > a FileWriter is open to it.
> >> > It is true that for  there is no way to get at the
> >> > underlying
> >> > File object.  That is perhaps a good thing for the use case of
> >> > downloading
> >> > to
> >> > a location specified by the user.
> >>
> >> Ah. But as far as I can tell (and remember), it's still fairly
> >> undefined what happens when the OS file under a File/Blob object is
> >> mutated.
> >>
> >> / Jonas
> >
> > Agreed.  I don't see it as a big problem.  Do you?  The application
> > developer is
> > in control.  They get to specify the output file (via FileWriter) that
> XHR
> > sends its
> > output to, and they get to know when XHR is done writing.  So, the
> > application
> > developer can avoid reading from the file until XHR is done writing.
>
> Well, it seems like a bigger deal here since the file is being
> constantly modified as we're downloading data into it, no? So for
> example if you grab a File object after the first progress event, what
> does that File object contain after the second? Does it contain the
> whole file, including the newly downloaded data? Or does it contain
> only the data after the first progress event? Or is the File object
> now invalid and can't be used?
>

What gears did about that was to provide a 'snapshot' of the
downloaded data each time responseBlob was called, with
the 'snapshot' being consistent with the progress events
having been seen by the caller. The 'snapshot' would remain
valid until discarded by the caller. Each snapshot just provided
a view onto the same data which maybe was in memory or
maybe had spilled over to disk unbeknownst to the caller.


>
> I'm also still unsure that a FileWriter is what you want generally. If
> you're just downloading temporary data, but data that happens to be so
> large that you don't want to keep it in memory, you don't want to
> bother the user asking for a location for that temporary file. Nor do
> you want that file to be around once the user leaves the page.
>

I think the point about not requiring the caller to manage the 'file' are
important.


> Sure, if the use case is actually downloading and saving a file for
> the user to use, rather than for the page to use, then a FileWriter
> seems like it would work. I.e. if you want something like
> "Content-Disposition: attachment", but where you can specify request
> headers. Is that the use case?
>

Mods to xhr to access the response more opaquely is a fairly general
feature request. One specific use case is to download a resource via xhr
and then save the results in a sandboxed file system. So "for the page to
use".

The notion of having a streaming interface on xhr is interesting. That with
a
BlobBuilder capability could work. If a streaming xhr mode provided new
data in the form of 'blobs' where each blob was just the newly received
data,
the caller could use a BlobBuilder instance to concatenate the set of
received
data blobs. And then take blobBuilder.getBlob() and do

Re: XMLHttpRequest.responseBlob

[Michael Nordman]

On Mon, Apr 26, 2010 at 4:24 PM, Jonas Sicking  wrote:

> On Mon, Apr 26, 2010 at 4:19 PM, Thomas Broyer  wrote:
> > On Tue, Apr 27, 2010 at 1:11 AM, Jonas Sicking  wrote:
> >>
> >> I'm not sure I understand how you envision the implementation working.
> >> You can't before hand know that the implementation won't ever access
> >> those properties (at least not without solving the halting problem).
> >> So you'll have to keep all the data in memory, just in case
> >> .reponseText is accessed.
> >
> > Couldn't you stream to disk, and in case responseText or responseXML
> > is called then read back from disk? You'd then have to "keep all the
> > data in memory" only when .responseText is used.
>
> That requires synchronous IO, something we're trying hard to avoid.
> Apple has said in the past that they are not willing to implement APIs
> that require synchronous IO. And in Firefox we're working on removing
> all places where it's used. Both in the implementation of various
> features exposed to web pages, but also in the product itself.
>

(geez halting problem)

Spill over to disk as a response gets "too big", a function of memory
available on the device. If a response gets "too big" its possibly being
swapped in VM anyway. You may know if its "too big" from the get go based on
content-length headers.

So if responseText is accessed on "too big" of a resource, yes that may
involve explicit file system IO (big deal). There's a trade off here...
cleaner API vs quirky API to make life easier for the user-agent. I was
leaning towards cleaner API.


>

/ Jonas
>

Re: XMLHttpRequest.responseBlob

[Michael Nordman]

On Mon, Apr 26, 2010 at 3:52 PM, Jonas Sicking  wrote:

> On Mon, Apr 26, 2010 at 3:39 PM, Darin Fisher  wrote:
> > On Mon, Apr 26, 2010 at 3:29 PM, Jonas Sicking  wrote:
> >>
> >> On Mon, Apr 26, 2010 at 3:21 PM, Darin Fisher 
> wrote:
> >> > There is some interest from application developers at Google in being
> >> > able
> >> > to get a Blob corresponding to the response body of a XMLHttpRequest.
> >> > The use case is to improve the efficiency of getting a Blob from a
> >> > binary
> >> > resource downloaded via XHR.
> >> > The alternative is to play games with character encodings so that
> >> > responseText can be used to fetch an image as a string, and then use
> >> > BlobBuilder to reconstruct the image file, again being careful with
> the
> >> > implicit character conversions.  All of this is very inefficient.
> >> > Is there any appetite for adding a responseBlob getter on XHR?
> >>
> >> There has been talk about exposing a responseBody property which would
> >> contain the binary response. However ECMAScript is still lacking a
> >> binary type.
> >>
> >> Blob does fit the bill in that it represents binary data, however it's
> >> asynchronous nature is probably not ideal here, right?
> >>
> >> / Jonas
> >
> >
> > I think there are applications that do not require direct access to the
> > response data.
> > For example,
> > 1- Download a binary resource (e.g., an image) via XHR.
> > 2- Load the resource using Blob.URN (assuming URN moves from File to
> Blob).
> > It may be the case that providing direct access to the response data may
> be
> > more
> > expensive than just providing the application with a handle to the data.
> >  Consider
> > the case of large files.
>
> Ah, so you want the ability to have the XHR implementation stream to
> disk and then use a Blob to read from there? If so, you need more
> modifications as currently the XHR implementation is required to keep
> the whole response in memory anyway in order to be able to implement
> the .responseText property.
>
> So we'll need to add some way for the page to indicate to the
> implementation "I don't care about the .responseText or .responseXML
> properties, just responseBlob"
>

Not sure we need additional API for that. If the page doesn't access those
properties, its indicating that it doesn't need them.


>
> / Jonas
>
>

Re: [IndexedDB] Granting storage quotas

[Michael Nordman]

On Wed, Apr 21, 2010 at 12:10 PM, Mike Clement  wrote:

> FWIW, the "transient" vs. "permanent" storage support is exactly why I
> eagerly await an implementation of EricU's Filesystem API.  Being able to
> guarantee that the UA will not discard potentially irreplaceable data is of
> paramount importance to web apps that want to work in an offline mode.
>
> I also find that the current arbitrary quota limit of 5MB per domain makes
> local storage APIs unusable for all but the most rudimentary apps (e.g.,
> sticky note demo apps).  There is an asymmetric distribution of local
> storage needs out there that no one is yet addressing (e.g., a photo- or
> video-related app might need GBs of local storage, an offline mail app might
> need tens or hundreds of MB, a TODO list app might only need kilobytes,
> etc.).
>
I wholeheartedly support any effort to coordinate quota management among all
> of the various local storage APIs.  The issue of quota limits is something
> that browser vendors will need to address soon enough, and it's probably
> best left up to them.  The need for "permanent" storage across all local
> storage APIs, though, is something that in my opinion should come out of the
> standardization process.
>

Here's a stab at defining programming interfaces that make a distinction
between "transient" vs "permanent" for the storage mechanisms. If we make
additions like this, we should use the same terminology across the board.

// WebSqlDBs, also could work for IndexedDBs
window.openDatabase(...);   // temporary
window.openPermanentDatabase(...);

// AppCaches, embellish the first line of the manifest file
CACHE MANIFEST
CACHE MANIFEST PERMANENT

// FileSystem, see the draft, i've change the terms a little here
window.requestFilesystem(...);// evictable
window.requestPermanentFilesystem(...)

// LocalStorage
window.localStorage;// purgeable
window.permanentLocalStorage;

Re: [IndexedDB] Granting storage quotas

[Michael Nordman]

On Tue, Apr 20, 2010 at 5:25 PM, Jeremy Orlow  wrote:

> On Tue, Apr 20, 2010 at 5:07 PM, Shawn Wilsher wrote:
>
>> On 4/20/2010 3:19 PM, Jeremy Orlow wrote:
>>
>>> This way of thinking is incompatible with offline web apps.  If I'm
>>> offline
>>> and I "send" and email, it needs to stay queued up to send until I'm
>>> reconnected to the internet.
>>>
>> I think a smart browser would include "am I offline" in it's heuristic for
>> granting storage space.
>
>
> Granting storage space is only part of the problem.
>
> Extrapolating from what you said, I guess I could see us keeping track of
> which origins have ever been accessed offline and making sure that data is
> never deleted without consulting the user.
>
> But, that doesn't solve this use case: something tike TiddlyWiki which
> lives totally on your system and is not supposed to be synced to the cloud.
>  The problem is that if you used this on your desktop, which is never
> offline, then the browser would not know it's precious.
>
> We could use the existence of an appCache manifest as a hint.  I guess that
> might be good enough.  But then we still have the malware problem.
>
>
>>  Anyone wanting to debate whether or not the UA should be free to clean
>>> up
>>> "persistent storage" without asking the user should read
>>>
>>> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-August/thread.html#22289
>>> and
>>> the various other threads it spawned first and only re-start things if
>>> they
>>> have new information to add.
>>>
>> I read it, but I don't see a consensus formed before it died off.  Am I
>> missing something?
>>
>
> It was kind of difficult to track.  The basic consensus was
> that persistent data can and should not be deleted without explicit user
> approval.
>
>
There was no such consensus. That was one position taken by some and not by
others.


>
>
> On Tue, Apr 20, 2010 at 5:18 PM, Nikunj Mehta  wrote:
>
>>
>> On Apr 20, 2010, at 3:19 PM, Jeremy Orlow wrote:
>>
>> This way of thinking is incompatible with offline web apps.  If I'm
>> offline and I "send" and email, it needs to stay queued up to send until I'm
>> reconnected to the internet.
>>
>>
>> I think the problem is that data loss could occur regardless of
>> "guarantees".
>>
>
> Sure, and that came up in the original giant "thread".  Even if something's
> in the cloud, it could be lost.  So what it really comes down to is best
> effort.
>
> Here are the classes of storage I hear you asking for:
>>
>> 1. temporary (no likelihood of data being being stored after session ends)
>>
>
> I don't think this type is very interesting.  The only use case is when you
> have so much data that the UA might need to spill to disk.  And if you're
> doing that, I'd imagine you'd want to use one of the other storage types
> anyway.
>
>
>> 2. evictable (no limit per site, except global limits, eviction candidates
>> chosen arbitrarily, including while application is running)
>> 3. non-evictable (no eviction, but data loss possible, limited by user)
>>
>
> These 2 are what we're talking about.
>

Re: [IndexedDB] Granting storage quotas

[Michael Nordman]

On Tue, Apr 20, 2010 at 5:18 PM, Nikunj Mehta  wrote:

>
> On Apr 20, 2010, at 3:19 PM, Jeremy Orlow wrote:
>
> This way of thinking is incompatible with offline web apps.  If I'm offline
> and I "send" and email, it needs to stay queued up to send until I'm
> reconnected to the internet.
>
>
> I think the problem is that data loss could occur regardless of
> "guarantees".
>
> Here are the classes of storage I hear you asking for:
>
> 1. temporary (no likelihood of data being being stored after session ends)
> 2. evictable (no limit per site, except global limits, eviction candidates
> chosen arbitrarily, including while application is running)
> 3. non-evictable (no eviction, but data loss possible, limited by user)
>
> Is this making sense?
>

That categorization does make sense, including the temporary.

I think we're (i know i am) most interested in the distinction between
evictable and non-evictable right now because the quota management and ui
implications. We've introduced these storage APIs that result in files on
disk, and now we want to put in some real-world storage management features.
The dumbed down 5MB per origin was a stop gap measure.

This is reminding me of Mike Wilson's  "state handling" post about different
scopes and lifetimes...
http://old.nabble.com/html5-state-handling:-overview-and-extensions-td24034773.html


>
> Anyone wanting to debate whether or not the UA should be free to clean up
> "persistent storage" without asking the user should read
> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-August/thread.html#22289
>  and
> the various other threads it spawned first and only re-start things if they
> have new information to add.
>
> J
>
> On Tue, Apr 20, 2010 at 3:10 PM, Nikunj Mehta  wrote:
>
>> As I see it, there's no such thing as "permanent" storage for Web browser
>> managed data. Even if a site expresses preferences that it would like to
>> keep its data resident for a long time, there cannot be a "guarantee" for
>> the data to be there permanently. If applications are bound to have to deal
>> with data disappearing while they are not running, we should not need to
>> spec browser behavior around the notion of purgeable or permanent.
>>
>> It makes sense for an application, OTOH, to say that it does not need data
>> to be stored on disk. IOW, create a database that is non-durable and, hence,
>> kept only in memory. Such databases are required to be empty upon creation.
>> They may be spilled over to disk, if implementations like to, but they will
>> not be retained from session to session.
>>
>> Nikunj
>>
>>
>> On Apr 20, 2010, at 2:37 PM, Michael Nordman wrote:
>>
>> I'd like to back up and challenge the notion of a per-site quota.
>>
>> In this discussion and others there is an underlying assumption that each
>> site has some well defined limit that the user-agent has granted it. I doubt
>> that's the best model. (Fyi: the chrome team's overly simplistic model
>> whereby each site gets 5M was not chosen because its a good model, this was
>> done just to proceed with building out the storage APIs independent of a
>> real storage management strategy).
>>
>> I'd like to set aside the per-site quota assumption and explore some
>> alternative models for web storage management.
>>
>> Some thoughts about the world we're designing for. There are an open ended
>> number of sites, each of which *could* use web storage in some form. From
>> that fact alone, it's impossible to come up with a quota that could be
>> granted to each and every site. It seems likely that the number of sites
>> that will actually require "permanent" storage is small compared to the
>> number of sites that *could* make use of more "volatile" storage, to borrow
>> jorlow's term, where the volatile data on disk can get scrapped by the
>> user-agent as needed. Maybe a better term for that class of storage is
>> "purgeable"?
>>
>> Maybe we should be designing for what seems to be the more common case,
>> lots of sites that make use of volatile/purgeable storage. But also come up
>> a means whereby the smaller number of sites that require stronger guarantees
>> can express the need for more permanent storage.
>>
>> "What if" by default all local storage is "purgeable". A lot of quota
>> issues melt away in this case since the user agent is free to reclaim
>> anything at anytime. I think it'd be reasonable if the user-agent never
>> asked 

Re: [IndexedDB] Granting storage quotas

[Michael Nordman]

On Tue, Apr 20, 2010 at 3:10 PM, Nikunj Mehta  wrote:

> As I see it, there's no such thing as "permanent" storage for Web browser
> managed data. Even if a site expresses preferences that it would like to
> keep its data resident for a long time, there cannot be a "guarantee" for
> the data to be there permanently. If applications are bound to have to deal
> with data disappearing while they are not running, we should not need to
> spec browser behavior around the notion of purgeable or permanent.
>


I see a difference between a cached version of a picture you've downloaded
vs a new picture taken while on vacation by a camera built into the device
and placed into a local repository managed by the user-agent. There is only
one copy of that picture in the world.

I'm looking for ways to make these storage APIs widely available w/o a lot
of user-prompting, but also for ways for webapps to express stronger
guarantees when needed. I think the notion of purgeable vs permanent may
help reconcile these conflicting goals.


> It makes sense for an application, OTOH, to say that it does not need data
> to be stored on disk. IOW, create a database that is non-durable and, hence,
> kept only in memory. Such databases are required to be empty upon creation.
> They may be spilled over to disk, if implementations like to, but they will
> not be retained from session to session.
>
> Nikunj
>
>
> On Apr 20, 2010, at 2:37 PM, Michael Nordman wrote:
>
> I'd like to back up and challenge the notion of a per-site quota.
>
> In this discussion and others there is an underlying assumption that each
> site has some well defined limit that the user-agent has granted it. I doubt
> that's the best model. (Fyi: the chrome team's overly simplistic model
> whereby each site gets 5M was not chosen because its a good model, this was
> done just to proceed with building out the storage APIs independent of a
> real storage management strategy).
>
> I'd like to set aside the per-site quota assumption and explore some
> alternative models for web storage management.
>
> Some thoughts about the world we're designing for. There are an open ended
> number of sites, each of which *could* use web storage in some form. From
> that fact alone, it's impossible to come up with a quota that could be
> granted to each and every site. It seems likely that the number of sites
> that will actually require "permanent" storage is small compared to the
> number of sites that *could* make use of more "volatile" storage, to borrow
> jorlow's term, where the volatile data on disk can get scrapped by the
> user-agent as needed. Maybe a better term for that class of storage is
> "purgeable"?
>
> Maybe we should be designing for what seems to be the more common case,
> lots of sites that make use of volatile/purgeable storage. But also come up
> a means whereby the smaller number of sites that require stronger guarantees
> can express the need for more permanent storage.
>
> "What if" by default all local storage is "purgeable". A lot of quota
> issues melt away in this case since the user agent is free to reclaim
> anything at anytime. I think it'd be reasonable if the user-agent never
> asked the user anything on a per-site basis. A user-agent could warn when
> system disk space crossed thresholds and give the user an option to set
> limits  on system disk space usage for webstorage as a whole.
>
> "What if" when creating local storage repositories (WebDBs or IndexedDBs or
> WebFileSystems or AppCaches) there was an optional means for the webapp to
> express "please consider this a permanent storage repository". The first
> time a site request "permanent" storage could be a reasonable time to
> interact with the user in some form, or to consult the user prefs about
> allowing permanent storage w/o being asked.
>
> I think ericu is baking in a distinction in between 'permanent' and
> 'temporary' in the FileSystem API he's working on. Some harmony across all
> flavors of local storage could be good.
>
> I actually think local storage management is an area where the webplatform
> has a chance to do a much better job then the desktop platforms have
> historically done. We don't need no stinking quotas ;) But we also don't
> need untold amounts of unused permanent storage littering disk drives
> needlessly around the globe (until the user gets a new system). A silly
> analogy. A computer is like a ship at sea. After years of usage, a whole
> bunch of barnacles build up on the hull and slow the vessel down. The
> webplatform to date is barnacle free in this area be

Re: [IndexedDB] Granting storage quotas

[Michael Nordman]

I'd like to back up and challenge the notion of a per-site quota.

In this discussion and others there is an underlying assumption that each
site has some well defined limit that the user-agent has granted it. I doubt
that's the best model. (Fyi: the chrome team's overly simplistic model
whereby each site gets 5M was not chosen because its a good model, this was
done just to proceed with building out the storage APIs independent of a
real storage management strategy).

I'd like to set aside the per-site quota assumption and explore some
alternative models for web storage management.

Some thoughts about the world we're designing for. There are an open ended
number of sites, each of which *could* use web storage in some form. From
that fact alone, it's impossible to come up with a quota that could be
granted to each and every site. It seems likely that the number of sites
that will actually require "permanent" storage is small compared to the
number of sites that *could* make use of more "volatile" storage, to borrow
jorlow's term, where the volatile data on disk can get scrapped by the
user-agent as needed. Maybe a better term for that class of storage is
"purgeable"?

Maybe we should be designing for what seems to be the more common case, lots
of sites that make use of volatile/purgeable storage. But also come up a
means whereby the smaller number of sites that require stronger guarantees
can express the need for more permanent storage.

"What if" by default all local storage is "purgeable". A lot of quota issues
melt away in this case since the user agent is free to reclaim anything at
anytime. I think it'd be reasonable if the user-agent never asked the user
anything on a per-site basis. A user-agent could warn when system disk space
crossed thresholds and give the user an option to set limits  on system disk
space usage for webstorage as a whole.

"What if" when creating local storage repositories (WebDBs or IndexedDBs or
WebFileSystems or AppCaches) there was an optional means for the webapp to
express "please consider this a permanent storage repository". The first
time a site request "permanent" storage could be a reasonable time to
interact with the user in some form, or to consult the user prefs about
allowing permanent storage w/o being asked.

I think ericu is baking in a distinction in between 'permanent' and
'temporary' in the FileSystem API he's working on. Some harmony across all
flavors of local storage could be good.

I actually think local storage management is an area where the webplatform
has a chance to do a much better job then the desktop platforms have
historically done. We don't need no stinking quotas ;) But we also don't
need untold amounts of unused permanent storage littering disk drives
needlessly around the globe (until the user gets a new system). A silly
analogy. A computer is like a ship at sea. After years of usage, a whole
bunch of barnacles build up on the hull and slow the vessel down. The
webplatform to date is barnacle free in this area because there are no
permanent local storage facilities... lets try to make these new features
not so barnacle prone too.

Cheers
-Michael

On Tue, Apr 20, 2010 at 11:17 AM, Shawn Wilsher  wrote:

> On 4/20/2010 4:11 AM, Mark Seaborn wrote:
>
>> 1) It doesn't allow a web app to ask for a storage allocation up front,
>> before it starts to consume the storage.
>>
> Why does that matter?
>
>
>  2) In Opera, the quota can only be increased in multiples of about 15, so
>> it
>> takes three prompts to get up into the range of gigabytes.
>>
> But there is an unlimited option, yeah?
>
>
>  3) The web app can't choose when the question is put to the user.
>> 4) The web app doesn't know how much storage has been allocated, so it
>> doesn't know when a question will be asked.
>> 5) In Opera, if the user chooses "Reject", they don't get prompted again.
>> This means that asking the user at an appropriate time is important for
>> the
>> continued functioning of the web app.  Prompting the user at the wrong
>> time
>> will interrupt them with a page-modal dialog which they might want to get
>> rid of with "Reject", which would potentially break the web app by leaving
>> it unable to get more storage.
>>
> These all feel like user-agent specific worries on how the user agent wants
> to bring this to the attention of the user.
>
> Cheers,
>
> Shawn
>
>

Re: FormData and BlobBuilder - duplication of functionality?

[Michael Nordman]

On Wed, Apr 14, 2010 at 2:19 PM, Jonas Sicking  wrote:

> On Wed, Apr 14, 2010 at 12:04 PM, Michael Nordman 
> wrote:
> > On Tue, Apr 13, 2010 at 9:01 PM, Jonas Sicking  wrote:
> >>
> >> On Tue, Apr 13, 2010 at 7:35 PM, Michael Nordman 
> >> wrote:
> >> > Good question. Let me ask you one. What value should you use for the
> >> > content-type header? That value needs to contain the boundary string.
> >> > You
> >> > need to know that to xhr.send the data in a way that looks like a form
> >> > submission. Just sending the blob will be "off by one" and the server
> >> > side
> >> > won't understand it.
> >>
> >> There seems to be general agreement (based on various discussions here
> >> and on whatwg) that the .type property should be moved to the Blob
> >> interface. So Blobs will have a content-type. So this problem should
> >> be taken care of.
> >>
> >> / Jonas
> >
> > Yes indeed!
> > So are we saying that FormData.toBlob() produces a blob representing the
> > encoded
> > results and having a .type property of the form...
> > multipart/form-data; boundary=xxx
>
> In the words of Samuel L Jackson: Correctamondo
>
> ;)
>
> > [Constructor] interface FormData { Blob toBlob (); void append(DOMString
> > name, Blob value); void append(DOMString name, DOMString value); };
> > Also it looks like BlobBuilder (in the draft dimich linked to) is lacking
> a
> > means for the caller to set the type attribute of the blob being built.
> > A couple ways that could be provided...
> > [Constructor] interface BlobBuilder { attribute DOMString endings;
> > attribute DOMString type; // option a
> > Blob getBlob (in DOMString type); // option b void append (in DOMString
> > text) raises (FileException); void append (in Blob data); };
>
> I don't feel strongly, but "option b" looks cleaner to me. Might want
> to make the argument optional though, and default to the empty string.
>

Option b works for me and agreed it should be optional with empty being the
default value.


>
> / Jonas
>

Re: FormData and BlobBuilder - duplication of functionality?

[Michael Nordman]

On Tue, Apr 13, 2010 at 9:01 PM, Jonas Sicking  wrote:

> On Tue, Apr 13, 2010 at 7:35 PM, Michael Nordman 
> wrote:
> > Good question. Let me ask you one. What value should you use for the
> > content-type header? That value needs to contain the boundary string. You
> > need to know that to xhr.send the data in a way that looks like a form
> > submission. Just sending the blob will be "off by one" and the server
> side
> > won't understand it.
>
> There seems to be general agreement (based on various discussions here
> and on whatwg) that the .type property should be moved to the Blob
> interface. So Blobs will have a content-type. So this problem should
> be taken care of.
>
> / Jonas
>

Yes indeed!

So are we saying that FormData.toBlob() produces a blob representing the
encoded
results and having a .type property of the form...
 multipart/form-data; boundary=xxx

[Constructor] interface FormData { Blob toBlob (); void append(DOMString
name, Blob value); void append(DOMString name, DOMString value); };

Also it looks like BlobBuilder (in the draft dimich linked to) is lacking a
means for the caller to set the type attribute of the blob being built.
A couple ways that could be provided...

[Constructor] interface BlobBuilder { attribute DOMString endings;
attribute DOMString type; // option a
Blob getBlob (in DOMString type); // option b void append (in DOMString
text) raises (FileException); void append (in Blob data); };

Re: FormData and BlobBuilder - duplication of functionality?

[Michael Nordman]

Good question. Let me ask you one. What value should you use for the
content-type header? That value needs to contain the boundary string. You
need to know that to xhr.send the data in a way that looks like a form
submission. Just sending the blob will be "off by one" and the server side
won't understand it.

Another way to deal with that would be to expose a FormData.boundary
attribute. In my made up use case, callers would have to store the boundary
out of band from the blob, and use it to make a properly formed content-type
header value when xhr.send'ing the blob.

Another reason isn't so much a "need"... but to make the programming
interface symmetrical and hopefully easier to understand and use.


On Tue, Apr 13, 2010 at 6:03 PM, Jonas Sicking  wrote:

> On Tue, Apr 13, 2010 at 5:53 PM, Michael Nordman 
> wrote:
> >
> >
> > On Tue, Apr 13, 2010 at 5:39 PM, Jonas Sicking  wrote:
> >>
> >> On Tue, Apr 13, 2010 at 5:34 PM, Michael Nordman 
> >> wrote:
> >> > Hmmm... maybe FormData could have a toBlob() method, and an inverse
> >> > means of
> >> > initializing a FormData object with a properly encoded Blob.
> >>
> >> What's the use case? ;)
> >
> > Persisting FormData and reconstituting it at a later time. For example
> when
> > offline then back online.
>
> Why would you need to recode into a FormData object though? You can
> submit a Blob using XHR the same way you can submit a FormData.


> / Jonas
>

Re: FormData and BlobBuilder - duplication of functionality?

[Michael Nordman]

On Tue, Apr 13, 2010 at 5:39 PM, Jonas Sicking  wrote:

> On Tue, Apr 13, 2010 at 5:34 PM, Michael Nordman 
> wrote:
> > Hmmm... maybe FormData could have a toBlob() method, and an inverse means
> of
> > initializing a FormData object with a properly encoded Blob.
>
> What's the use case? ;)
>

Persisting FormData and reconstituting it at a later time. For example when
offline then back online.

/ Jonas

Re: FormData and BlobBuilder - duplication of functionality?

[Michael Nordman]

FormData produces multipart encoded data (with boundaries and such) whereas
BlobBuilder simply concatenates data together. Those are different
functions. Having different ctors for these object types
seems appropriate to me. There's also been talk of either initializing a
 with FormData and/or extracting FormData from a . If things
like that come to be, a different object type helps with clarity (at least
to me).

Hmmm... maybe FormData could have a toBlob() method, and an inverse means of
initializing a FormData object with a properly encoded Blob.

On Tue, Apr 13, 2010 at 5:11 PM, Dmitry Titov  wrote:

> Hi,
>
> It seems we are currently implementing 2 very similar objects - 
> FormData
>  and BlobBuilder.
> Both can be created, then strings or Blobs can be appended, then it can be
> sent via XHR or saved into a file (well, FormData can not be saved in the
> file yet). But, they are very similar:
>
> var bb = new BlobBuilder();
> bb.appendText("Lorem ipsum");
> bb.appendBlob(someBlob);
> xhr.send(bb.getBlob());
>
> var fd = new FormData();
> fd.appendText("data1", "Lorem ipsum");
> fd.appendBlob("data2", someBlob);
> xhr.send(fd);
>
> Are those two similar enough to justify merging them into a single object
> (defined as a a list of named BlobItems) that could be used for both?
>
> One difference is that BlobBuilder does not use a mime wrapper around data
> pieces, but rather concatenates them - there could be a property for that.
>
> Dmitry
>

Re: Not making partial result available during FileReader.readAsText()?

[Michael Nordman]

Seems pretty clear from the snippet you provided, it says you SHOULD provide
partially decoded results in the result attribute as progress is made.

On Thu, Apr 8, 2010 at 8:31 PM, Jian Li  wrote:

> For FileReader.readAsText, the spec seems to allow partial file data being
> decoded and saved in the result attribute when progress event is fired:
>
> Make progress notifications. As the bytes from the fileBlob argument are
> read, user agents SHOULD ensure that on getting, the result attribute
> returns partial file data representing the number of bytes currently loaded
> (as a fraction of the total) [ProgressEvents], decoded in memory according
> to the encoding determination.
>
>
> The partial file data read so far might not get decoded completely. Could
> we choose not to decode the partial result till we retrieve all the data,
> just like what FileReader.readAsDataURL does?
>
>

Re: [FileAPI] Blob.URN?

[Michael Nordman]

On Tue, Mar 30, 2010 at 4:56 PM, Darin Fisher  wrote:

> The only way to get a FileWriter at the moment is from  type="saveas">.  What is desired is a way to simulate the load of a resource
> with Content-Disposition: attachment that would trigger the browser's
> download manager.


When it comes to questions like these (when offline, how to do something
that happens in the online case)... i tend to say why not do it exactly the
same way as its done in the online case... so load a resource with a
Content-Disposition:attachment header on it :)


>
> -Darin
>
>
>
> On Tue, Mar 30, 2010 at 4:32 PM, Jonas Sicking  wrote:
>
>> What's the use case for specifying the Content-Disposition mime type.
>> The ones I've heard so far seems better solved using the FileWriter
>> [1] API.
>>
>> [1] http://dev.w3.org/2009/dap/file-system/file-writer.html
>>
>> / Jonas
>>
>> On Tue, Mar 30, 2010 at 4:22 PM, Michael Nordman 
>> wrote:
>> > There is more than just the mime type when dealing with the URLs.
>> > There at least two content headers of interest, Content-Type
>> > and Content-Disposition.
>> > Whatever mechanism involved should allow for both of these content
>> headers
>> > to be set
>> > by the web application.
>> >
>> > On Tue, Mar 30, 2010 at 3:07 PM, Jonas Sicking 
>> wrote:
>> >>
>> >> On Tue, Mar 30, 2010 at 2:57 PM, Darin Fisher 
>> wrote:
>> >> > On Tue, Mar 30, 2010 at 12:22 AM, Jonas Sicking 
>> >> > wrote:
>> >> >>
>> >> >> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher 
>> >> >> wrote:
>> >> >> > Apologies if this has been discussed before, but I'm curious why
>> URN
>> >> >> > is
>> >> >> > not
>> >> >> > a property of Blob.  It seems like it would be useful to be able
>> to
>> >> >> > load
>> >> >> > a
>> >> >> > slice of a File.  For example, this could be used by an
>> application
>> >> >> > to
>> >> >> > fetch
>> >> >> > all of its subresources out of a single file.
>> >> >>
>> >> >> IIRC originally it was placed on File since Blobs do not have a
>> >> >> content type. However I think there is general agreement that it
>> >> >> should be moved to Blob.
>> >> >>
>> >> >> However it would be great to be able to assign a content type to a
>> >> >> Blob. Possibly slice() could take a optional argument.
>> >> >>
>> >> >
>> >> > Adding an optional parameter to slice() sounds attractive indeed.
>> >> > BlobBuilder [1] should probably also have such an optional argument.
>> >>
>> >> Indeed!
>> >>
>> >> / Jonas
>> >>
>> >
>> >
>>
>
>

Re: [FileAPI] Blob.URN?

[Michael Nordman]

Ah... the "save as" ability associated with the FileWriter API does change
the equation a bit. My main motivation for setting Content-Disposition was
to trigger the user-agents download manager when navigating to these urls
instead of trying to display them in a browser frame.


On Tue, Mar 30, 2010 at 4:32 PM, Jonas Sicking  wrote:

> What's the use case for specifying the Content-Disposition mime type.
> The ones I've heard so far seems better solved using the FileWriter
> [1] API.
>
> [1] http://dev.w3.org/2009/dap/file-system/file-writer.html
>
> / Jonas
>
> On Tue, Mar 30, 2010 at 4:22 PM, Michael Nordman 
> wrote:
> > There is more than just the mime type when dealing with the URLs.
> > There at least two content headers of interest, Content-Type
> > and Content-Disposition.
> > Whatever mechanism involved should allow for both of these content
> headers
> > to be set
> > by the web application.
> >
> > On Tue, Mar 30, 2010 at 3:07 PM, Jonas Sicking  wrote:
> >>
> >> On Tue, Mar 30, 2010 at 2:57 PM, Darin Fisher 
> wrote:
> >> > On Tue, Mar 30, 2010 at 12:22 AM, Jonas Sicking 
> >> > wrote:
> >> >>
> >> >> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher 
> >> >> wrote:
> >> >> > Apologies if this has been discussed before, but I'm curious why
> URN
> >> >> > is
> >> >> > not
> >> >> > a property of Blob.  It seems like it would be useful to be able to
> >> >> > load
> >> >> > a
> >> >> > slice of a File.  For example, this could be used by an application
> >> >> > to
> >> >> > fetch
> >> >> > all of its subresources out of a single file.
> >> >>
> >> >> IIRC originally it was placed on File since Blobs do not have a
> >> >> content type. However I think there is general agreement that it
> >> >> should be moved to Blob.
> >> >>
> >> >> However it would be great to be able to assign a content type to a
> >> >> Blob. Possibly slice() could take a optional argument.
> >> >>
> >> >
> >> > Adding an optional parameter to slice() sounds attractive indeed.
> >> > BlobBuilder [1] should probably also have such an optional argument.
> >>
> >> Indeed!
> >>
> >> / Jonas
> >>
> >
> >
>

Re: [FileAPI] Blob.URN?

[Michael Nordman]

There is more than just the mime type when dealing with the URLs.
There at least two content headers of interest, Content-Type
and Content-Disposition.
Whatever mechanism involved should allow for both of these content headers
to be set
by the web application.


On Tue, Mar 30, 2010 at 3:07 PM, Jonas Sicking  wrote:

> On Tue, Mar 30, 2010 at 2:57 PM, Darin Fisher  wrote:
> > On Tue, Mar 30, 2010 at 12:22 AM, Jonas Sicking 
> wrote:
> >>
> >> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher 
> wrote:
> >> > Apologies if this has been discussed before, but I'm curious why URN
> is
> >> > not
> >> > a property of Blob.  It seems like it would be useful to be able to
> load
> >> > a
> >> > slice of a File.  For example, this could be used by an application to
> >> > fetch
> >> > all of its subresources out of a single file.
> >>
> >> IIRC originally it was placed on File since Blobs do not have a
> >> content type. However I think there is general agreement that it
> >> should be moved to Blob.
> >>
> >> However it would be great to be able to assign a content type to a
> >> Blob. Possibly slice() could take a optional argument.
> >>
> >
> > Adding an optional parameter to slice() sounds attractive indeed.
> > BlobBuilder [1] should probably also have such an optional argument.
>
> Indeed!
>
> / Jonas
>
>

Re: [FileAPI] Blob.URN?

[Michael Nordman]

There is more than just the mime type when dealing with the URLs.
There at least two content headers of interest, Content-Type
and Content-Disposition.
Whatever mechanism involved should allow for both of these content headers
to be set
by the web application.


On Tue, Mar 30, 2010 at 3:07 PM, Jonas Sicking  wrote:

> On Tue, Mar 30, 2010 at 2:57 PM, Darin Fisher  wrote:
> > On Tue, Mar 30, 2010 at 12:22 AM, Jonas Sicking 
> wrote:
> >>
> >> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher 
> wrote:
> >> > Apologies if this has been discussed before, but I'm curious why URN
> is
> >> > not
> >> > a property of Blob.  It seems like it would be useful to be able to
> load
> >> > a
> >> > slice of a File.  For example, this could be used by an application to
> >> > fetch
> >> > all of its subresources out of a single file.
> >>
> >> IIRC originally it was placed on File since Blobs do not have a
> >> content type. However I think there is general agreement that it
> >> should be moved to Blob.
> >>
> >> However it would be great to be able to assign a content type to a
> >> Blob. Possibly slice() could take a optional argument.
> >>
> >
> > Adding an optional parameter to slice() sounds attractive indeed.
> > BlobBuilder [1] should probably also have such an optional argument.
>
> Indeed!
>
> / Jonas
>
>

Re: [FileAPI] Blob.URN?

[Michael Nordman]

> Wouldn't it be better to have a constructor for File. File(Blob, name,
type, contentdisposition).

That could work, not sure its as intuitive. I think Files are destined to be
more often returned by various APIs and less often constructed by
application code directly. Wrapping a File/Blob in another File in order to
'override' whatever content-headers are baked into the original feels less
direct then specifying how you want the browser to view the File when given
a particular URL.

Mutable properties of the File object would be confusing. For example, does
setting the 'name' attribute rename the underlying file or tweeking type put
a new extension on the file name? The answer is no, but its easy to see how
somebody may expect that behavior.

I'd also like to point out that a getURL() method with an option to specify
the content -headers is compatible with a .url attribute that produces a URL
that will result in 'default' content-headers for the underlying File.

> Also, didn't we decide to change URN to URL? As far as I can tell that is
how Gecko is implementing it.

That may be, I dug this snippet up from an discussion long ago.


On Wed, Mar 24, 2010 at 1:58 AM, Anne van Kesteren  wrote:

> On Wed, 24 Mar 2010 03:40:36 +0100, Michael Nordman 
> wrote:
>
>> This has been discussed before, not sure what the conclusion was (if any)
>> http://www.mail-archive.com/public-webapps@w3.org/msg06137.html
>>
>> <http://www.mail-archive.com/public-webapps@w3.org/msg06345.html>
>>
>>
>> In order for the URN to be useful, it would have to have a mediaType
>> associated with it, and then there's content-disposition to think
>> about, which then wants a file name as well...boy, that's a lot of
>> baggage.  However, since these aren't really inherent properties of
>> the Blob, just of the way you want the browser to view the Blob, it
>> would seem natural to me do to something like this:
>>
>>interface Blob {
>>  ...
>>  DOMString getURN(in DOMString mediaType,
>>   [Optional] in DOMString contentDisposition,
>>   [Optional] in DOMString name);
>>};
>>
>> 
>>
>
> Wouldn't it be better to have a constructor for File. I.e.
>
>  File(Blob, name, type, contentdisposition)
>
> or some such. (Maybe some of the attributes on File should be made mutable,
> i.e. name and mime...)
>
>
> Also, didn't we decide to change URN to URL? As far as I can tell that is
> how Gecko is implementing it.
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: [FileAPI] Blob.URN?

[Michael Nordman]

This has been discussed before, not sure what the conclusion was (if any)
http://www.mail-archive.com/public-webapps@w3.org/msg06137.html



In order for the URN to be useful, it would have to have a mediaType
associated with it, and then there's content-disposition to think
about, which then wants a file name as well...boy, that's a lot of
baggage.  However, since these aren't really inherent properties of
the Blob, just of the way you want the browser to view the Blob, it
would seem natural to me do to something like this:

interface Blob {
  ...
  DOMString getURN(in DOMString mediaType,
   [Optional] in DOMString contentDisposition,
   [Optional] in DOMString name);
};




On Tue, Mar 23, 2010 at 7:19 PM, Dmitry Titov  wrote:

> Blob would need a content-type for that, but it could probably easy be
> added as a property that is assignable.
>
> BTW if the Blob could have a urn and mime type, this info could be directly
> used to form the headers for the Blob when sending it in multipart form
> using FormData.
>
> Dmitry
>
> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher  wrote:
>
>> Apologies if this has been discussed before, but I'm curious why URN is
>> not a property of Blob.  It seems like it would be useful to be able to load
>> a slice of a File.  For example, this could be used by an application to
>> fetch all of its subresources out of a single file.
>>
>> -Darin
>>
>
>

Re: [WebSQLDatabase] Adding a vacuum() call

[Michael Nordman]

Instead of calling back on success only, maybe call back on completion
regardless of success or failure. This way the caller would know when the
potentially lengthy operation was done, regardless of the outcome.

2010/3/11 Dumitru Daniliuc 

> joao,
>
> it looks like we mostly agree on this feature, so i was wondering if we
> could formally agree on a spec. here's what i propose:
>
> 1. name: vacuum. to the best of my knowledge, all current WebSQLDatabases
> implementations use SQLite, and in SQLite the command is called VACUUM. so
> it seems to me that we might as well call the new function vacuum(). what do
> you think?
>
> 2. spec: no need for an error callback.
>
>
> interface Database {
>   // the methods and properties currently in the spec
>   void vacuum(in optional SQLVoidCallback successCallback);
> };
>
> 3. what the call should do: the purpose of this call is to allow apps to
> vacuum/compact/defragment/clean up their databases whenever they see fit. a
> call to vacuum() could take a considerable amount of time (especially on
> larger or highly fragmented databases); therefore, it is not recommended for
> web apps to call this method during periods of high activity.
>
> how to process a vacuum() call:
>
>1. if the UA does not support this call (mobile browsers?), jump to
>step 3.
>2. queue up a task to vacuum/compact/defragment/clean up the database.
>3. if the task succeeds, and a success callback is provided, queue up a
>task to invoke the success callback; in all other cases (the task failed, 
> or
>no success callback was provided), do nothing: proceed to the next task in
>the queue.
>
> does this seem acceptable? we (google engineers interested in this) feel
> that UAs should either not implement the vacuum() call, or they should
> respect it (rather than taking it as a hint). it is ok for UAs to keep track
> of things like system idleness or databases closing to do more vacuuming
> that the apps asked for, if they want to. however, we feel that a vacuum()
> request by an app should not be postponed, simply because sometimes apps
> know better than UAs when the best time to vacuum is (it might be nice to
> give apps more information on how fragmented their databases are, but that's
> a separate discussion).
>
> thanks,
> dumi
>
>
> 2010/3/9 Jeremy Orlow 
>
> On Mon, Mar 8, 2010 at 8:47 PM, Dumitru Daniliuc  wrote:
>>
>>>
>>>
>>> On Mon, Mar 8, 2010 at 3:39 AM, João Eiras  wrote:
>>>

  I don't see how the callbacks are useful though. Vacuum works
>> transparently, its effects are not visible, and what should the page
>> do in
>> case of error ?
>>
>>
> i was thinking of something like:
>
> db.defragment(errorCallback, successCallback);
> showAPrettyImageAndAskTheUserToWait();
>
> function errorCallback(error) {}
> function successCallback() {
>  getRidOfThePrettyImageAndRestartTheApp();
> }
>
> just like you, i'm not sure if the error callback is useful at all, but
> i
> thought i'd add it to make the defragment() call look more like a
> transaction. maybe we don't need it.
>
>
 True, but this is a kind of operation that could very well just run on
 the background, with a single optional callback when it's done (the webpage
 can't do anything if an error is detected anyway).
>>>
>>>
>>> ok, so let's drop the errorCallback: vacuum([optional] successCallback);
>>>
>>>
 The user agent would need to queue any subsequent transactions if a
 vacuum is running. I would consider it as an hint, and after all webpages
 that own references to the underlying data files are closed, would do a
 vacuum. So, if you have many tabs on gmail, and that a single gmail 
 instance
 tries to do multiple vacuums, it would equiv to one single vacuum 
 operation.
>>>
>>>
>>> what do we do if some databases are opened for the entire life of the
>>> browser? for example, i open my browser which has myfavoriteapp.com set
>>> as its homepage. myfavoriteapp.com immediately opens a DB, and i only
>>> close that app when i close the browser. when would the browser vacuum
>>> myfavoriteapp's DBs in this case?
>>>
>>> i think it's ok for the UA to vacuum some DBs automatically when it
>>> thinks it's a good time to do so; however, if a platform supports the
>>> vacuum/defrag call (i.e. if it doesn't treat it is a no-op), then i think a
>>> vacuum call coming from the app should be immediately scheduled (yes, the
>>> subsequent transactions would have to wait for the vacuuming to finish
>>> running). in some cases, the apps know better than the UA when to vacuum
>>> their DBs.
>>>
>>> by the way, we should probably agree on a name for this call. which one
>>> do you prefer? vacuum, defragment, defrag, something else? i don't have a
>>> strong opinion.
>>>
>>
>> I think vacuum is fine since the spec is already tied to the SQLite SQL
>> dialect.
>>
>> collectGarbage() is 

Re: FormData questions

[Michael Nordman]

Horray for FormData! My $0.02

* UTF8 always

* Sending initiates the POST of a snapshot of the FormData at the time send
is called, subsequent modifications to the FormData don't affect what is
asyncly being sent, but do affect subsequent sends of the modified FormData.

* XHR.send(formData) always sets the response Content-Type including the
boundary tag used for the multipart encoding, overwriting any previously set
value. Hmmm... perhaps send() should throw an exception if there is a
previously set header value to make it more clear that callers shouldn't be
setting the Content-Type when sending() this class of object.

On Sun, Feb 14, 2010 at 7:32 PM, Jonas Sicking  wrote:

> On Sun, Feb 14, 2010 at 6:04 PM, Dmitry Titov  wrote:
> >
> >
> > On Sat, Feb 13, 2010 at 6:44 PM, Jonas Sicking  wrote:
> >>
> >> On Sat, Feb 13, 2010 at 6:02 PM, Dmitry Titov 
> wrote:
> >> > On Fri, Feb 12, 2010 at 5:32 PM, Jonas Sicking 
> wrote:
> >> >>
> >> >> Hi WebApps fans!
> >> >>
> >> >> Working on implementing FormData and ran into a couple of questions.
> >> >>
> >> >> First of all, I assume that it is intended that a FromData object can
> >> >> be submitted several times. I.e. that the following code is ok:
> >> >>
> >> >> fd = new FormData;
> >> >> fd.append("name1", "foo");
> >> >> xhr1 = new XMLHttpRequest;
> >> >> xhr1.open(...);
> >> >> xhr1.send(fd);
> >> >> fd.append("name2", "bar");
> >> >> xhr2 = new XMLHttpRequest;
> >> >> xhr2.open(...);
> >> >> xhr2.send(fd);
> >> >>
> >> >> where the first XHR will send 1 name/value pair, and the second XHR
> >> >> will send 2. I do think this should be allowed, but I wanted to make
> >> >> sure others agreed.
> >> >
> >> > What can be a reason to disallow this? FormData is just a collection
> of
> >> > data
> >> > objects. So assuming those XHR objects are sync, the code should work
> as
> >> > you
> >> > described.
> >>
> >> It complicates implementation a tiny amount, but IMHO not enough to
> >> disallow it.
> >>
> >> > Interesting question though - what happens if XHR is async and the
> >> > content
> >> > of FormData changes while async operation is in progress. By analogy
> >> > with
> >> > scenario when underlying file of Blob object changes while async
> reading
> >> > operation is in progress, would it be reasonable to fail the send and
> >> > return
> >> > 'error' ProgressEvent?
> >>
> >> I don't think raising an 'error' event should be correct, no. In my
> >> example above I think the two requests should succeed successfully,
> >> and the first one should submit one name/value pairs, and the second
> >> should submit two.
> >
> > Does it mean that implementation should basically produce actual form
> data
> > synchronously (do a deep copy) at the moment of xhr.send() even if the
> xhr
> > is asynchronous? In case FormData includes Blobs backed by files on the
> disk
> > it may be prohibitive.
>
> I believe the text (non-Blob) data should be synchronously copied,
> yes. The Blob data needs no special handling beyond what you already
> have to do.
>
> The way we implement this in Firefox is that we create a multiplex
> data stream which consists of textual data mixed with Blob-backed
> data. So we create a stream implementation that will alternately read
> from in memory textual data, alternately from one or more Blobs.
>
> The object we create synchronously upon the call to send() just
> contains the text data and pointers to various Blobs. While this
> object is created synchronously, no data is read from the Blob. Once
> the network code reads from the stream, data is asynchronously read
> from the Blob.
>
> Don't know what setup you have for normal form submissions, so I'm not
> sure if you can duplicate this behavior exactly.
>
> Another way to look at it is that for the following code:
>
> fd = new FormData;
> fd.append("foo", "bar");
> fd.append("name", myFile);
>
> This synchronously creates a FormData object that contains enough
> information to submit both textual data and a Blob. And since you can
> create that information synchronously, you should for the following
> code
>
> xhr.send(fd);
>
> be able to synchronously create a clone of the fd that similarly
> contains enough information to submit both textual and Blob data, and
> then set up to transmit that clone.
>
> / Jonas
>
>

Re: File API: Blob and underlying file changes.

[Michael Nordman]

On Thu, Jan 21, 2010 at 12:49 PM, Jonas Sicking  wrote:

> One thing to remember here is that if we require snapshotting, that
> will mean paying potentially very high costs every time the
> snapshotting operation is used. Potetially copying hundreds of
> megabytes of data (think video).
>
>
I was thinking of different semantics. If the underlying bits change
sometime after a 'snapshot' is taken, the 'snapshot' becomes invalid and you
cannot access the underying bits. If an application wants guaranteed access
to the 'snapshot', it would have to explicitly save a copy somewhere
(sandboxed file system / coin a new transient 'Blob' via a new blob.copy()
method) and refer to the copy.

So no costly copies are made w/o explicit direction to do so from the app.

But if we don't require snapshotting, things will only break if the
> user takes the action to modify a file after giving the page access to
> it.
>
> Also, in general snapshotting is something that UAs can experiment
> with without requiring changes to the spec. Even though File.slice is
> a synchronous function, the UA can implement snapshotting without
> using synchronous IO. The UA could simply do a asynchronous file copy
> in the background. If any read operations are performed on the slice
> those could simply be stalled until the copy is finished since reads
> are always asynchronous.
>
> / Jonas
>
> On Thu, Jan 21, 2010 at 11:22 AM, Eric Uhrhane  wrote:
> > On Thu, Jan 21, 2010 at 11:15 AM, Jian Li  wrote:
> >> Treating blobs as snapshots sounds like a reasonable approach and it
> will
> >> make the life of the chunked upload and other scenarios easier. Now the
> >> problem is: how do we get the blob (snapshot) out of the file?
> >> 1) We can still keep the current relationship between File and Blob.
> When we
> >> slice a file by calling File.slice, a new blob that captures the current
> >> file size and modification time is returned. The following Blob
> operations,
> >> like slice, will simply inherit the cached size and modification time.
> When
> >> we access the underlying file data in XHR.send() or FileReader, the
> >> modification time will be verified and an exception could be thrown.
> >
> > This would require File.slice to do synchronous file IO, whereas
> > Blob.slice doesn't do that.
> >
> >> 2) We can remove the inheritance of Blob from File and introduce
> >> File.getAsBlob() as dimich suggested. This seems to be more elegant.
> >> However, it requires changing the File API spec a lot.
> >>
> >> On Wed, Jan 20, 2010 at 3:44 PM, Eric Uhrhane  wrote:
> >>>
> >>> On Wed, Jan 20, 2010 at 3:23 PM, Dmitry Titov 
> wrote:
> >>> > On Wed, Jan 20, 2010 at 2:30 PM, Eric Uhrhane 
> wrote:
> >>> >>
> >>> >> I think it could.  Here's a third option:
> >>> >> Make all blobs, file-based or not, just as async as the blobs in
> >>> >> option 2.  They never do sync IO, but could potentially fail future
> >>> >> read operations if their metadata is out of date [e.g. reading
> beyond
> >>> >> EOF].  However, expose the modification time on File via an async
> >>> >> method and allow the user to pass it in to a read call to enforce
> >>> >> "fail if changed since this time".  This keeps all file accesses
> >>> >> async, but still allows for chunked uploads without mixing files
> >>> >> accidentally.  If we allow users to refresh the modification time
> >>> >> asynchronously, it also allows for adding a file to a form, changing
> >>> >> the file on disk, and then uploading the new file.  The user would
> >>> >> look up the mod time when starting the upload, rather than when the
> >>> >> file's selected.
> >>> >
> >>> > It would be great to avoid sync file I/O on calls like Blob.size.
> They
> >>> > would
> >>> > simply return cached value. Actual mismatch would be detected during
> >>> > actual
> >>> > read operation.
> >>> > However then I'm not sure how to keep File derived from Blob, since:
> >>> > 1) Currently, in FF and WebKit File.fileSize is a sync I/O that
> returns
> >>> > current file size. The current spec says File is derived from Blob
> and
> >>> > Blob
> >>> > has Blob.size property that is likely going to co-exist with
> >>> > File.fileSize
> >>> > for a while, for compat reasons. It's weird for file.size and
> >>> > file.fileSize
> >>> > to return different things.
> >>>
> >>> True, but we'd probably want to deprecate file.fileSize anyway and
> >>> then get rid of it, since it's synchronous.
> >>>
> >>> > 2) Currently, xhr.send(file) does not fail and sends the version of
> the
> >>> > file
> >>> > that exists somewhere around xhr.send(file) call was issued. Since
> File
> >>> > is
> >>> > also a Blob, xhr.send(blob) would behave the same which means if we
> want
> >>> > to
> >>> > preserve this behavior the Blob can not fail async read operation if
> >>> > file
> >>> > has changed.
> >>> > There is a contradiction here. One way to resolve it would be to
> break
> >>> > "File
> >>> > is Blob" and to be able to "capture the File as Blob" 

Re: Renaming WebDatabase and WebSimpleDB

[Michael Nordman]

Web-Indexed-Storage

On Mon, Nov 30, 2009 at 4:52 PM, Nikunj R. Mehta wrote:

>
> On Nov 30, 2009, at 3:14 PM, Michael(tm) Smith wrote:
>
>  Jeremy Orlow , 2009-11-30 14:46 -0800:
>>
>>  I agree with Mike, but I'd also note that "Web Key-Value Database" could
>>> easily be confused with WebStorage given that it also uses a Key-Value
>>> model.
>>>
>>
>> True but we know the distinction is that Web Storage does not use
>> a database.
>>
>
>
> Do we make naming decisions considering just us WG members as its audience
> or that of the general public? I think the general public is well within its
> rights to treat Web Storage as a persistence technology that seems to be
> like "Key-Value" database.
>
> I want to emphasize here that I think "key-value" in the title misses the
> subtlety - it is the use of index sequential access that is at the heart of
> WebSimpleDB, and not key-value storage.
>
>
> Nikunj
> http://o-micron.blogspot.com
>
>
>
>
>

Re: [webworkers] SharedWorker and ApplicationCache

[Michael Nordman]

Shared workers do not depend on HTML documents for resource loading. I think
the webkit impl may have it cobbled together that way at the moment, but
thats per-happen-stance, not per-the-spec.

Shared workers effectively establish a new top-level-browsing-context all
unto themselves.

On Mon, Nov 9, 2009 at 10:09 AM, Alexey Proskuryakov  wrote:

>
> 07.11.2009, в 10:47, Michael Nordman написал(а):
>
>
>  I've been wondering if SharedWorkers should have a means of establishing
>> an appcache similar to how pages can via the  mechanism.
>>
>> My mental model is that a shared worker is very much like a top-level page
>> with respect to appcaches, but that means for a shared worker to
>> express/establish its appcache is missing.
>>
>
> Don't shared workers depend on HTML documents for network loading? I'm not
> sure how they can have independent caches, if they just ask a document to
> fetch a resource for them.
>
> - WBR, Alexey Proskuryakov
>
>

Re: [webworkers] SharedWorker and ApplicationCache

[Michael Nordman]

I was thinking something more in parallel with how HTML pages establish the
appcache association. There are very precise algorithms that define how this
cache association is to be performed for 'master-entry' pages. I think ll of
those algorithms could apply to shared worker scripts, if only shared worker
scripts possessed a means to declare an association with a manifest file.
What is missing is a means for the worker script to declare that
association.

Pages can do this:   

Shared workers have no such syntax.

Its not up to the invoker of a page to express this association, its up to
the author of the page do to so. I'd vote to do the same for shared workers.
Otherwise, if we create a different means of declaring the association, a
different set of algorithms have to be specified to handle it.

I can think of at least two ways for the author of the worker script to
express it's manifest file.

1. Blatantly hacky, comment-based syntax

// SharedWorker.applicationCache.manifest = 'foo'

2. A script-based syntax, a new attribute in the workerContext for shared
workers.

workerContext.applicationCache.manifest = 'foo';

// the provided value is interpretted as relative to the shared worker
script url
// can only be set when initially eval'ing the script into scope
(otherwise throws)
// can only be set once (otherwise throws)
// must be set prior to initiation of any sub-resource loads (otherwise
throws)
// can be read anytime and will reflect the string value set by the
caller (if any)


On Sat, Nov 7, 2009 at 12:00 PM, Drew Wilson  wrote:

> Yeah, I thought about this some back when I was trying to piece together a
> proposal for persistent workers. I suppose you could pass an optional
> manifest URL to the SharedWorker constructor, with appropriate restrictions
> on different pages creating the same SharedWorker but with different
> manifest URLs. Since there's already an optional name parameter, you could
> always require the name to be specified if you are specifying an app cache
> URL (so the third argument to the constructor would always be the manifest
> URL).
>
> -atw
>
>
> On Sat, Nov 7, 2009 at 10:47 AM, Michael Nordman wrote:
>
>> I've been wondering if SharedWorkers should have a means of establishing
>> an appcache similar to how pages can via the  mechanism.
>>
>> My mental model is that a shared worker is very much like a top-level page
>> with respect to appcaches, but that means for a shared worker to
>> express/establish its appcache is missing.
>>
>>
>> On Sat, Nov 7, 2009 at 9:14 AM, Drew Wilson  wrote:
>>
>>> You may have two separate pages running from different app caches (or
>>> version), each of which is trying to access the same shared worker, so we
>>> don't want to tie it explicitly to a specific parent's app cache/version.
>>>
>>> It does seem a bit wonky, though - if you have one parent who has an app
>>> cache that has two resources in it (a.js and b.js) and another parent who
>>> has an app cache that has a different two resources in it (a.js and c.js),
>>> it's non-deterministic which app cache the shared worker would be associated
>>> with, and this could break apps.
>>>
>>> I'm not sure that there's a good solution for this, given that manifests
>>> can only be associated with an HTML resource.
>>>
>>> -atw
>>>
>>>
>>> On Sat, Nov 7, 2009 at 8:57 AM, Anne van Kesteren wrote:
>>>
>>>> We were wondering why there is a quite complicated resolution algorithm
>>>> to determine the ApplicationCache that belongs to the SharedWorker rather
>>>> than just passing the ApplicationCache to the SharedWorker at creation time
>>>> (i.e. as constructor argument). Is there anything that is gained by the
>>>> current model?
>>>>
>>>>
>>>> --
>>>> Anne van Kesteren
>>>> http://annevankesteren.nl/
>>>>
>>>>
>>>
>>
>

Re: [webworkers] SharedWorker and ApplicationCache

[Michael Nordman]

I've been wondering if SharedWorkers should have a means of establishing an
appcache similar to how pages can via the  mechanism.

My mental model is that a shared worker is very much like a top-level page
with respect to appcaches, but that means for a shared worker to
express/establish its appcache is missing.


On Sat, Nov 7, 2009 at 9:14 AM, Drew Wilson  wrote:

> You may have two separate pages running from different app caches (or
> version), each of which is trying to access the same shared worker, so we
> don't want to tie it explicitly to a specific parent's app cache/version.
>
> It does seem a bit wonky, though - if you have one parent who has an app
> cache that has two resources in it (a.js and b.js) and another parent who
> has an app cache that has a different two resources in it (a.js and c.js),
> it's non-deterministic which app cache the shared worker would be associated
> with, and this could break apps.
>
> I'm not sure that there's a good solution for this, given that manifests
> can only be associated with an HTML resource.
>
> -atw
>
>
> On Sat, Nov 7, 2009 at 8:57 AM, Anne van Kesteren wrote:
>
>> We were wondering why there is a quite complicated resolution algorithm to
>> determine the ApplicationCache that belongs to the SharedWorker rather than
>> just passing the ApplicationCache to the SharedWorker at creation time (i.e.
>> as constructor argument). Is there anything that is gained by the current
>> model?
>>
>>
>> --
>> Anne van Kesteren
>> http://annevankesteren.nl/
>>
>>
>

Re: Value of Server-Sent Events

[Michael Nordman]

On Sat, Oct 24, 2009 at 9:37 PM, Ian Hickson  wrote:

> On Sat, 24 Oct 2009, Michael Nordman wrote:
> > On Fri, Oct 23, 2009 at 8:45 PM, Ian Hickson  wrote:
> > > On Fri, 23 Oct 2009, Michael Nordman wrote:
> > > >
> > > > An area that may be worth exploring, that would add to the list
> > > > things that go beyond syntactic sugar, could be for multiple
> > > > documents to listen in on the same event-stream backed by the same
> > > > connection to the server. This could reduce the total number of
> > > > hanging GET style connections used by an application (spread across
> > > > multiple pages / workers) on a particular client, and by extension
> > > > the number of connections seen by the server-side.
> > >
> > > The spec technically allows this already, though as written it
> > > requires sending all the same messages again, so it's not really
> > > practical (and loses the "don't have to hang on to everything"
> > > advantage). It's not clear to me how we would know which messages are
> > > skippable -- maybe in v2 we can add a flag that indicates that this
> > > message block should be remembered, and another flag to indicate that
> > > all previous remembered messages should be dropped, or something like
> > > that, and when a new page hooks in, if the event source supports it,
> > > it would just get the saved messages and then resume with whatever the
> > > next message is.
> >
> > > sending all the same messages again... flag that indicates...
> remembered
>
> "sending all the same messages again" may have been a poor use of words. I
> meant that all the events already received by the browser on the existing
> connection would be (re)dispatched to the new EventSource object. I agree
> that's suboptimal; to get around that we'd need a way for the server to
> opt-in to a system whereby people can join in half-way and only get new
> messages. To do that, though, while still supporting the case of some
> messages being critical (e.g. a message saying how the rest of the
> messages are to be decompressed), we'd need a mechanism to indicate
> "critical" messages that are to be stored and dispatched to any new
> clients when a new EventSource object to that URL is created.
>

I'm not sure this would really need a notion of "critial" messages. That may
just
be complicating things unnecessarily.

If an application is sending data with those kind of characteristics, it
could be handled
at the application level. It can send a "keyFrame" bit of its own and drop
non key frames on the client side until it comes across the first key it
observes.

> a way for the server to opt-in to a system whereby people can join
> in half-way and only get new messages

Here's the mechanism for that purpose i was hinting at...

I don't know what the current content type is defined for event streams, but
suppose its...
Content-Type: application/event-stream // the one defined for
non-shareable streams
Content-Type: application/event-stream-shareable

A shareable stream cannot have "critical" messages (as far as the user-agent
knows). The user agent,
and any network appliances between the origin-server and that user-agent,
need not buffer anything.

--

Perhaps some scoping of how widely "shareable" a stream is could make
sense...

// the one defined for non-shareable streams
Content-Type: application/event-stream

// can be shared by everyone (network appliances could mux these streams)
Content-Type: application/event-stream-universally-shareable

// can only be shared within a particular session
Content-Type: application/event-stream-session-shareable

The latter could be useful for event-streams that contain data specific to a
particular authenticated user.







>
> --
> Ian Hickson   U+1047E)\._.,--,'``.fL
> http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
> Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
>

Re: Value of Server-Sent Events

[Michael Nordman]

On Fri, Oct 23, 2009 at 8:45 PM, Ian Hickson  wrote:

> On Fri, 23 Oct 2009, Michael Nordman wrote:
> >
> > An area that may be worth exploring, that would add to the list things
> > that go beyond syntactic sugar, could be for multiple documents to
> > listen in on the same event-stream backed by the same connection to the
> > server. This could reduce the total number of hanging GET style
> > connections used by an application (spread across multiple pages /
> > workers) on a particular client, and by extension the number of
> > connections seen by the server-side.
>
> The spec technically allows this already, though as written it requires
> sending all the same messages again, so it's not really practical (and
> loses the "don't have to hang on to everything" advantage). It's not clear
> to me how we would know which messages are skippable -- maybe in v2 we can
> add a flag that indicates that this message block should be remembered,
> and another flag to indicate that all previous remembered messages should
> be dropped, or something like that, and when a new page hooks in, if the
> event source supports it, it would just get the saved messages and then
> resume with whatever the next message is.
>

> sending all the same messages again... flag that indicates... remembered

That's certainly not what I had in mind? I'll try to describe more clearly
what I was getting at with a simple contrived use-case. There's nothing
complicated about it.

There's a thermometer somewhere in the world connected to a web application
server. The web application server responds to two HTTP GET requests.

GET /current_temp
A short-lived request that returns a textual response that looks like {
"temp": 22.22, "time": "" }.

GET /temp_stream
This one is intended for use with a long-lived hanging GET (ala
event-stream). It returns a temp/time sample once every 60 seconds in the
same json format.

When a page opens the /temp_stream event-stream, it only receives temp/time
samples that occur after its having joined the party. This model wouldn't
work for all event-stream use cases, but for some it would.

How to distinguish which event-streams were "shareable" or not is an
interesting question? Maybe that's most properly a function of the
Content-Type of the event-stream?


> --
> Ian Hickson   U+1047E)\._.,--,'``.fL
> http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
> Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
>

Re: Value of Server-Sent Events

[Michael Nordman]

On Fri, Oct 23, 2009 at 3:21 PM, Maciej Stachowiak  wrote:

>
> On Oct 23, 2009, at 1:44 PM, Jonas Sicking wrote:
>
>
>> I also continue to miss actual developer demand for server sent
>> events. Seems like it doesn't add a lot of sugar over simply using
>> XMLHttpRequest and progress events. But again, I'm fine with
>> publishing a new WD.
>>
>
> Besides syntactic sugar, here are some advantages over using XHR and
> progress events (or readystatechange events, which some client-side JS
> libraries use today):
>
> - Does not force the full event stream to be stored in memory until the
> connection is closed (XHR's responseText effectively forces this). This is
> the biggest one. A long event stream shouldn't take progressively more
> memory.
>
> - Does not force you to reparse the event stream each time new data comes
> in (XHR + progress events doesn't have an easy way to get just the new data
> chunk).
>
> - Dealing with message boundaries instead of packet boundaries (or
> arbitrary batching-within-the-network-layer boundaries) is way more
> convenient and likely more efficient to a degree that I think goes beyond
> mere syntactic sugar.
>
> Regards,
> Maciej
>
>
I buy all of those advantages. This feature is a nice formalization of the
commonly used "hanging GET" found in many ajax applications.

An area that may be worth exploring, that would add to the list things that
go beyond syntactic sugar, could be for multiple documents to listen in on
the same event-stream backed by the same connection to the server. This
could reduce the total number of hanging GET style connections used by an
application (spread across multiple pages / workers) on a particular client,
and by extension the number of connections seen by the server-side.

Re: HTML extension for system idle detection.

[Michael Nordman]

This particular proposal is clearly a good feature and Bjoern's is a good
articulation of privacy concerns.

Something lacking in the web platform in general (to my knowledge, albeit
limited) is a permissioning scheme whereby sites or applications can
establish rights. Is there any work being done on that dimension that has
any traction? Provided mechanisms for sites/applications to request
(declaratively or programatically), and for users to acquiesce (and to later
reject), a great many APIs could be introduced to the web platform. The
specter of 'privacy' or 'security' concerns could be more easily addressed
with such mechanisms.

The proposed idle state monitoring interfaces could be opaquely presented to
an unauthorized site as...
 systemIdleState() == UNAUTHORIZED
 systemIdleTimeInSeconds() == 0
... and no events ever fire

nit: systemIdleState() and systemIdleTimeInSeconds() feel more like a
read-only attributes than functions

On Thu, Sep 17, 2009 at 6:34 PM, Bjoern Hoehrmann  wrote:

> * Jeremy Orlow wrote:
> >As far as I know, there really aren't any.  This was discussed on WhatWG
> >(before being directed here) and IIRC there were no serious security or
> >privacy concerns.  The minimum resolution of the event makes attacks based
> >on keystroke timing impossible.  Some people suggested that web apps could
> >do something "bad" while the user is away, but I don't think anyone could
> >come up with a good example of something "bad".  Can you think of any
> >specific concerns?
>
> If you consider a client-server instant messaging service, it is easy to
> give three examples for why you would not want the server and peers be
> informed whether you're currently interacting with the device the client
> is running on out of "privacy" considerations.
>
> If the peer's client indicates that you are using the system, then it is
> common for peers to assume you are actually present and not merely cause
> some activity every now and then (e.g., change the volume setting while
> watching a movie, check on some activity while cleaning the house) and
> to become upset if you do not respond. Similarily, you may be available
> but do not cause system activity (e.g., watch a movie, but the client is
> configured to interrupt the playback on receiving a message) and peers
> are likely to incorrectly assume you are absent and not contact you.
>
> You may also be present for all intents and purposes, but do not wish to
> give some of your peers the impression you are (e.g., you may not wish
> to attend to them that instant). Similar are finer grained notifications
> of user activity like typing notifications. If the client transmits them
> you may start typing in some message, reconsider, and then might have to
> answer the peer's question what you wanted to say but did not.
>
> Prolonged storage of this information also allows for analysis of beha-
> vioral patterns; for example, if the user is rarely inactive for certain
> periods of time, that is likely to be seen as an indication of a medical
> condition such as a sleep disorder.
>
> I believe there are plenty of users of instant messaging systems who
> have turned off, or would turn off if they were aware of the option and
> possible consequences, these kind of notifications for these and other
> reasons, or adjust their behavior to avoid the possible consequences.
> --
> Björn Höhrmann · mailto:bjo...@hoehrmann.de · http://bjoern.hoehrmann.de
> Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
> 25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/
>
>

Re: Proposal for sending multiple files via XMLHttpRequest.send()

[Michael Nordman]

On Mon, Sep 14, 2009 at 1:26 PM, Jonas Sicking  wrote:

> 2009/9/14 Michael Nordman :
> > What I think we really want is a script-only means of sending multipart
> > form-data encoding POSTs that contain a mix of file- parts and
> binary-parts
> > (in addition to the ability to send the raw contents of a file).
> > * script-only, so these POSTs can be performed in workers
> > * multipart form-data, its the defacto standard
> > The Gears stab at this was BlobBuilder, which let developers compose
> > something with a mix of utf8 encoded strings and binary ile data, and
> > Gears.HttpRequest.send(blob). Developers were left to their own devices
> to
> > produce a valid mulitpart form-data encoded blob by manually stitching
> > together the parts with appropiate boundaries and such, and then setting
> the
> > content-type on the request properly prior to sending. Somewhat tedious
> for
> > developers but it worked.
>
> I definitely think that we need to add some way of sending a stream
> that is a concatenation of strings, binary data, and files, without
> requiring the files to be read.
>
> Once we have that, there's a question of if we need to add convenience
> features for specific formats, such as multipart form-data and/or
> JSON.
>

Agreed... and gears stopped at the first milestone you described without
adding the convenience classes.


>
> / Jonas
>

Re: Proposal for sending multiple files via XMLHttpRequest.send()

[Michael Nordman]

What I think we really want is a script-only means of sending multipart
form-data encoding POSTs that contain a mix of file- parts and binary-parts
(in addition to the ability to send the raw contents of a file).
* script-only, so these POSTs can be performed in workers
* multipart form-data, its the defacto standard

The Gears stab at this was BlobBuilder, which let developers compose
something with a mix of utf8 encoded strings and binary ile data, and
Gears.HttpRequest.send(blob). Developers were left to their own devices to
produce a valid mulitpart form-data encoded blob by manually stitching
together the parts with appropiate boundaries and such, and then setting the
content-type on the request properly prior to sending. Somewhat tedious for
developers but it worked.

The Gears team had considered expressing things as an array with a mix of
strings and Blobs, and then sending the array as well, but shied away from
that because we .send([]) already has a meaning (as has been pointed out in
this thread).

Another way to expose this capability in a script-only way could be the
introduction of a new object for this purpose.

class FormData {
  addString(name, value);
  addFile(name, file);  // maybe optional params for additional
content-disposition headers?
}
xhr.send(formData);






2009/9/12 Alfonso Martínez de Lizarrondo 

> > Since XMLHttpRequest spec has already added the overload for
> send(document),
> > why not just adding more overload for file and array of items? IMHO,
> having
> > similar send*** methods, like sendFile, together with overloads of send()
> > might make the API more complicated.
>
> If the browser doesn't provide any system to detect that it supports
> this feature, then the only way to enable it is to do UA sniffing, and
> I thought that everyone agreed that this is bad because when the code
> is deployed it must be updated for each new browser that implements
> this API.
>
> I wrote about my point of view about this kind of features in
> http://alfonsoml.blogspot.com/2009/08/upload-progress-for-firefox-35.html
> As I said there, if other browser implements that API the code will
> work automatically for them. It's not a problem to add new code to
> support a new API, but make it work based on UA it isn't nice.
>
>

Re: [WebSimpleDatabase] New spec, editor's draft available

[Michael Nordman]

Try this link instead...http://dev.w3.org/2006/webapi/WebSimpleDatabase/
Nikunj's link actually points to http://dev.w3.org/2006/webapi/DataCache/ which
is also interesting, but seemingly not what was intended.

@Nikunj, I have not yet read thru your draft in detail, but will do so. In
principle, I agree with the notion of a standard that does not depend on a
particular sql dialect (or an assumed sql dialect) to reap the rewards of
structured storage, query, and retrieval. As I said, I haven't read it thru
yet... have you addressed full text indexing and search in some form? And if
not, that would be a feature request.


On Fri, Sep 4, 2009 at 1:45 PM, Nikunj R. Mehta wrote:

> I have published the first draft of the WebSimpleDatabase API, which is
> based on B-tree databases [1]. Here's a link to the draft:
>
> http://dev.w3.org/2006/webapi/WebSimpleDatabase/
>
> Abstract:
> This document defines APIs for storing and retreving ordered
> key-value pairs in a transactional database.
>
> I would request feedback that we can incorporate before publication as a
> working draft. Please note that certain items are purposely under-specified
> at the moment. I want to gauge the potential interest in this direction and
> evolve it to meet the needs of the WG.
>
> Nikunj
> http://o-micron.blogspot.com
>
> [1] http://www.w3.org/mid/f480f60a-5dae-4b73-922a-93ed401cf...@oracle.com
>

Re: [web databases] changeVersion error reporting

[Michael Nordman]

If the version is stored in the database file itself, as is with webkit's
impl for example, the database object does not have immediate access to that
value.
On Thu, Sep 3, 2009 at 5:57 PM, João Eiras  wrote:

> Hi!
>
> Database.changeVersion expects oldVersion and newVersion arguments. The
> Database also specifies a version attribute.
> The transaction steps tell to fail the transaction due to the failed
> preflight condition if oldVersion does not match the current database
> version.
> Meanwhile, the callbacks have been made optional, and rightly so.
>
> So:
>  - if the author does not use the callbacks, there no error reporting that
> the preflight condition failed
>  - the version check can be done synchronously when changeVersion is called
> because the Database object has immediate access to that value, and it's
> much more convenient for the author to just get an exception (like
> INVALID_STATE_ERR) than to have to go through the entire callback process
> just to realize later that the transaction failed due to the mismatched
> version. It also spares the user agent from creating and executing a
> transaction that WILL fail.
>
> To solve these issues, I would suggest doing immediate synchronous
> validation of the version when changeVersion is called, throwing an
> exception on error, and drop that step from the preflight operations.
>
> Thank you.
>
>
>
>

Re: [webdatabase] Transaction Locks

[Michael Nordman]

   1.

   Open a new SQL transaction to the database, and create a
SQLTransaction
object
   that represents that transaction. If the *mode* is read/write, the
   transaction must have an exclusive write lock over the entire database. If
   the *mode* is read-only, the transaction must have a shared read lock
   over the entire database. The user agent should wait for an appropriate lock
   to be available.
   2.

   If an error occurred in the opening of the transaction (e.g. if the user
   agent failed to obtain an appropriate lock after an appropriate delay), jump
   to the last step.
   3.

   If a *preflight operation* was defined for this instance of the
   transaction steps, run that. If it fails, then jump to the last step. (This
   is basically a hook for the
changeVersion()
   method.)
   4.

   Queue a task to invoke the *transaction callback* with the
   aforementioned
SQLTransaction
object
   as its only argument, and wait for that task to be run.

10. (last) Queue a task to invoke the *error callback* with a newly
constructed SQLError  object
that represents the last error to have occurred in this transaction.
Rollback the transaction. Any still-pending statements in the transaction
are discarded.

I think its clear what the intent is. A failure to acquire the lock (BEGIN
fails) leads to an error callback w/o having called the transaction
callback. At least thats my reading of it.

On Mon, Aug 31, 2009 at 4:37 AM, Lachlan Hunt wrote:

> Hi,
>  In the processing model [1], step 2 says:
>
>  "If an error occurred in the opening of the transaction (e.g. if the
>   user agent failed to obtain an appropriate lock after an appropriate
>   delay), jump to the last step."
>
> It's not clear if the spec requires the transaction to fail to open in the
> case that it can't yet obtain an appropriate lock, or whether the spec just
> allows that as an implementation decision.
>
> According to our developer, the way we've implemented it is that we will
> always create a new transaction and run the transaction callback, but the
> SQL statements themselves will be queued up and run whenever the lock
> becomes available.  There is no timeout that will cause it to invoke the
> error callback.
>
> Is this acceptable, or should the transaction callback not be run while
> there is another lock in effect on the database?
>
> [1] http://dev.w3.org/html5/webdatabase/#processing-model
>
> --
> Lachlan Hunt - Opera Software
> http://lachy.id.au/
> http://www.opera.com/
>
>

Re: Alternative File API

[Michael Nordman]

Currently we have File and FileData, where a File object ISA FileData
object... completion is delivered via readXXX method completion callbacks.
It would be easy to add progress callbacks to those readXXX methods.
I think Garrets point about two different corners of the webapp issuing
reads against a File/FileData at the same time really only becomes an issue
when one of those two corners decides to cancelReads() on the File/FileData
instance.

Strictly speaking, I think the seperate 'Reader' class makes for a more
correct API. The two corners above would not conflict since each would
presumably be working with a distinct FileReader object. And the seperate
'Reader' with event handlers seems more javscript'y. Your code snippets
demonstrate that it takes more busy work to use... but setting the event
handlers has to happen somewhere.

If we go with distinct 'Readers', maybe have the 'Data' object responsible
for manufacturing them as there could be differences in where the 'Data'
really resides and what it takes to retrieve it.

var reader = data.createReader();

File - represents a file in the file system
BinaryData - represents a bag of binary bits (roughly analogous to a
Gears.Blob)
BinaryDataReader - an interface to read those bits

File isa BinaryData
XHR.ResponseBody isa BinaryData
SQLBlob isa BinaryData

Michael

On Sat, Aug 15, 2009 at 9:19 AM, Garrett Smith wrote:

> On Tue, Aug 11, 2009 at 7:20 PM, Jonas Sicking wrote:
> > Here is an alternative proposal for an API for reading files:
> >
>
> [snip proposal]
>
> >
> > As stated, I'm not convinced that this is a better solution than what
> > the spec currently does. The only advantage I can see is that it
> > supports progress events without requiring the use of XMLHttpRequest,
> > and the downside is that it's a significantly bigger API. Usage
> > examples would be:
> >
>
> The advantage to having the callback as an event is that it lets
> multiple callbacks be attached easily to the read.
>
> The File can be found in the DOM as:-
>
> var files = input.files,
>file =  files && files[0];
>
> That file might be gotten from two separate unrelated parts of the
> code. Each could a callback to that file and issue different getXXX
> commands to it, creating a race condition whereby one callback could
> get called where it was expecting the payload to have text and another
> callback to get called where it was expecting a payload of binary
> text.
>
> Instead, a separate reader can be obtained to read the file and that
> reader can have the callback.
>
> > Current draft:
> > myFile.getAsBinaryString(handler);
> > function handler(data, error) {
> >  doStuffWith(data);
> > }
> >
> > Above API:
>
> // Is it "FileRequest" or "FileReader"?
> > reader = new FileReader;
>
> // The following two statements are backwards.
> > reader.readAsBinaryString(myFile);
> > reader.onload = handler;
> > function handler(event) {
> >  doStuffWith(event.target.response);
> > }
> >
>
> What happens when the reader is in process of reading?
> var reader = new FileReader;
> reader.onload = handler;
> reader.readAsBinaryString(myFile);
> reader.readAsText(myFile);
>
> The callback would have to know in advance what type of read happened.
> So you'd want to have a different reader for each type of read. For
> example:-
>
> var bReader = new FileReader;
> bReader.onload = handler;
> bReader.readAsBinaryString(myFile);
> var tReader = new FileReader;
> tReader.readAsText(myFile);
>
> As you can see, the read /type/ is exclusive to the reader. "tReader"
> is only reading "text" and "bReader" is only reading binary. Each
> reader reads only one type. So doesn't a Factory seem more appropriate
> than a bunch of constructors?
>
> var bReader = FileReader.create(FileReader.BINARY); // (the "etc" part).
> bReader.onload = handler;
> bReader.read(myFile);
>
> > / Jonas
>
> Garrett
>
>

Re: RFC: WebApp timing

[Michael Nordman]

interface Window {

  readonly attribute Timing

pageTiming;

};

The pageTiming attribute represents the timing information related to the
> current top-level browsing context. Each top-level browser context must have
> a unique pageTiming attribute.


You mention top-level browsing context specifically, not just browsing
context.

* Firstly, why not just 'browsing context'? Seems like these timings also
apply to subframe navigations.

* Secondly, does that imply that accessing window.pageTiming from a nested
frame returns the timing for its 'top' page? Or perhaps an 'empty' timing
object or 'null'?

>

Re: [File API] feedback on August 1/5 draft

[Michael Nordman]

On Fri, Aug 7, 2009 at 3:21 PM, Jonas Sicking  wrote:

> On Fri, Aug 7, 2009 at 12:23 PM, Michael Nordman
> wrote:
> >> > I think getAsURL() should become an attribute instead. E.g.
> >> >
> >> >  readonly attribute DOMString localURL;
> >> >
> >> > Since it is just a reference there is no need this needs to be
> >> > asynchronous and there is also no need for it to be method.
> >>
> >> Agreed.
> >
> > The draft says a new UUID should be 'coined' for each method invocation.
> > (Why is that?) Given the coinage of a new url on each access, accessing
> it
> > thru an attribute feels a little odd.
>
> I think we should removed that requirement though.


I'd be in favor of removing that requirement too.

(Unless there is an actual reason for it to start with? The draft was very
expicit about calling for a new guid on each access, i have no clue why.)


>
>
> / Jonas
>

Re: FileAPI "splice" method

[Michael Nordman]

agreed... slice() seems more appropiate

On Wed, Aug 5, 2009 at 10:34 PM, Adam de Boor  wrote:

> this is a minor point, but I'm finding the name of the "splice" method to
> be odd. To me splice means to join, and "slice" would seem a more
> appropriate name. The Array object has both splice and slice, and the former
> is used for removing and inserting data and modifies the array in place,
> while the latter pulls out a sub-part of the array into a new array.
> a
>
> --
> Adam de Boor
> Google
>

Re: [File API] feedback on August 1/5 draft

[Michael Nordman]

On Thu, Aug 6, 2009 at 11:29 AM, Jonas Sicking  wrote:

> On Thu, Aug 6, 2009 at 4:04 AM, Anne van Kesteren wrote:
> > Thanks for the update to the draft! Below some feedback:
> >
> > In the table of contents the link to the filedata URL scheme is broken.
> >
> >
> > The Web IDL syntax needs to be updated. E.g. FileList can be simply
> described as
> >
> >  typedef sequence FileList;
>
> You mean:
>
> typedef sequence FileList;
>
> right?
>
> > I have not received any feedback on my comments as to why getAsDataURL is
> actually needed. I still think it should be dropped.
>
> Given that the filedata url is very limited in time, getAsDataURL
> still seems useful IMHO.
>
> > I think getAsURL() should become an attribute instead. E.g.
> >
> >  readonly attribute DOMString localURL;
> >
> > Since it is just a reference there is no need this needs to be
> asynchronous and there is also no need for it to be method.
>
> Agreed.


The draft says a new UUID should be 'coined' for each method invocation.
(Why is that?) Given the coinage of a new url on each access, accessing it
thru an attribute feels a little odd.


>
>
> > The constants of FileError need to be actually placed on the FileError
> object and renumbered as to make sense. They are not DOM exceptions so it
> does not make sense to align with that in any way.
>
> It seems useful to use the same code for people that want to display
> error messages to the user, this way you can either pass the value in
> the DOM event or from an exception to the same function.
>
> I also can't see a downside to aligning the values?
>
> But I don't feel strongly here.
>
> > Last time I also made comments regarding the details of discovering the
> encoding of a file etc. Those still seem to apply.
>
> Got a pointer to the actual question?
>
> / Jonas
>
>

Re: DataCache API - editor's draft available

[Michael Nordman]

This is very encouraging. It sounds like we agree in principal that the best
place for the features represented in the DataCache proposal are extensions
to the AppCache corner of the world.

My preference would be to keep the official AppCache spec stable right now,
but to begin designing those extensions sooner rather than later. This would
help inform the architecture of AppCache implementations underway by browser
vendors today, and help better inform application teams of what to expect as
well. The DataCache proposal is a great starting point.

In a sense, specify these extensions on a "development branch" of the spec.


On Wed, Jul 22, 2009 at 10:56 PM, Ian Hickson  wrote:

> On Wed, 22 Jul 2009, Adrian Bateman wrote:
> >
> > My preference would be to see this functionality proposed as an
> > evolution of AppCache. While I can't commit that we would implement it
> > any time soon, it would be included in our considerations and at the
> > very least if we implement AppCache we would try to ensure any
> > architecture we select wouldn't preclude these extensions in the future.
> > With work on the HTML5 spec trying to get it locked down towards a Last
> > Call, adding new features to that document is clearly less desirable and
> > I understand Ian's reluctance to incorporate extensions there.
>
> My preference in general would be for us to wait until we have stable,
> well-tested implementations before adding more features to appcache, but I
> certainly think that on the long term, appcache should be extended to
> support more than it does now. As it stands, the model is quite
> extensible, so I think it would be relatively easy for us to move in that
> direction in the future.
>
> --
> Ian Hickson   U+1047E)\._.,--,'``.fL
> http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
> Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
>
>

Re: DataCache API - editor's draft available

[Michael Nordman]

On Wed, Jul 22, 2009 at 4:00 PM, Maciej Stachowiak  wrote:

>
> On Jul 21, 2009, at 11:25 PM, Nikunj R. Mehta wrote:
>
>  On Jul 21, 2009, at 9:15 PM, Adrian Bateman wrote:
>>
>>  While it might not be the perfect solution (we know the web far from
> ideal and is a lot of compromise), this type of proposal would be a
> lot more compelling to me if I could say "This is what we have to
> add, this is how, and here are the use cases that make it valuable"
> with a roadmap for extending what people are already building
> instead of something brand new.
>

 Would you mind explaining the last point "with a roadmap for extending
 what people are already building instead of something brand new." for
 me? I would definitely strive to make the proposal more compelling.

>>>
>>> What I'm asking for is a more unified proposal that says "If you have
>>> already implemented AppCache, here's what you add to make the same cache
>>> provide the additional functionality needed to enable these additional use
>>> cases." This will inevitably be a compromise from what a pure implementation
>>> looks like (your current DataCache spec, say) but lots of the web is
>>> necessarily a compromise because it builds on prior art that might not have
>>> been ideal but has been specified, built and deployed (and not always in
>>> that order).
>>>
>>> This would allow people to form a judgement about whether the additional
>>> use cases are worth the additional effort instead of whether the additional
>>> use cases are worth yet another cache. I think the ship has already sailed
>>> on AppCache and we can't undo that even if we wanted to and I don't think a
>>> competing solution is the right approach.
>>>
>>
>> What kind of extensions/changes to AppCache would be acceptable at this
>> point? In a previous exchange with Ian, he declined to consider DataCache
>> like extensions to ApplicationCache for HTML5, which might be the reasonable
>> thing to do. I can of course put together a proposal, but it would be good
>> to know from browser vendors what their preferences are in this matter.
>>
>> I am open to the idea of incrementally evolving AppCache to be more
>> supportive of DataCache requirements.
>>
>
> We'd be willing to consider implementing AppCache extensions in WebKit even
> if they were initially proposed in a separate specification from HTML5,
> assuming there was rough consensus among browser vendors that the extensions
> are a good idea.
>
> I think the ability to have a resource that is served by a client-side
> script would be an example of a reasonable extensions. I'm not totally sure
> how to recast all of the DataCache ideas to fit with the AppCache model. I'd
> be glad to review and provide suggestions, if everyone thinks this is a good
> idea.


Thank you for putting this proposal together. The Chrome team is willing as
well to consider adding extensions along these lines to Chrome. And several
app teams here at Google would be willing to put them to good use.


>
> Regards,
> Maciej
>
>
>

Re: File API Feedback

[Michael Nordman]

On Wed, Jul 22, 2009 at 4:44 AM, Anne van Kesteren  wrote:

> On Wed, 22 Jul 2009 04:07:18 +0200, Arun Ranganathan 
> wrote:
> > This still sends strings, but XHR 2[2] has got:
> >
> >   void send(in ByteArray data);
> >
> > which makes me think we need a binary getter as well :)
>
> Yes, it is called responseBody at the moment. I intent to replace both with
> support for FileData (the new Blob as I understand it) whenever that is
> defined and published.


Another aspect of the use case we're discussing (composing a
multipart/form-data POST for sending vis XHR2) is NOT having to actually
read the full contents of the file parts into memory while doing this
composition. That wouldn't work for a large file or for a large number of
smaller files.

Gears BlobBuilder accomplishes that by constructing a resulting Blob that
internally keeps track of a collection of constituent Blobs (some of which
may be memory resident and other backed by a files on disk). When Gears
HttpRequest sends one of these Blobs, it reads things incrementally. So no
big memory usage spikes when dealing with file backed Blobs during
composition of the multipart Blob or during the sending of it (also no big
disk usage spikes during composition or sending).



>
>
>
> > [2] http://www.w3.org/TR/XMLHttpRequest2/#xmlhttprequest
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: File API Feedback

[Michael Nordman]

http://code.google.com/apis/gears/api_blobbuilder.html

> This I'm less sanguine about, since the immediately desired capability is>
to work with existing system files.
The motivating usecase for this feature in Gears is to compose the body of a
POST in the multipart/form-data format, including binary file parts. The
resulting blobs can then be uploaded with HttpRequest. For example, Gmail
uses this when sending/saving messages with attachments that had been queued
for delivery while offline.

Does the FileAPI + XHR2 provide for anything like that?

Re: File API Feedback

[Michael Nordman]

The BlobBuilder.append() method is central to the use-case I'm referring to.
builder.append(string data that comprises the multipart/form-data header);
builder.append(string data that comprises simple text part);
builder.append(string data that comprises the start of a binary part);
builder.append(fileBlob); // the file itself
builder.append(string data that closes the binary part);
// perhaps more binary and text parts
builder.append(string data that closes the multipart/form-data);
multipartBlob = builder.getAsBlob();

httpRequest.send(multipartBlob);

> For now, can a byte-ranged splice method defined on FileData (to
asynchronously create "ranged" FileData objects) satisfy your use case?

Splice doesn't come into play for the use case described above. The
motivation for the .splice method is to "chunk" very large files into many
parts, and to upload them individually... think large YouTube videos, where
an upload craps out after 80% has been done... you end up with nothing. By
splicing and uploading in smaller bites, the app is free to implement a
strategy where its not all-or-nothing. If any individual chunk fails, resend
that chunk.



On Tue, Jul 21, 2009 at 3:44 PM, Arun Ranganathan  wrote:

> Michael Nordman wrote:
>
>> http://code.google.com/apis/gears/api_blobbuilder.html
>>
>>
>>
>>> This I'm less sanguine about, since the immediately desired capability
>>> is>
>>>
>>>
>> to work with existing system files.
>>
>>
>
>  The motivating usecase for this feature in Gears is to compose the body of
>> a
>> POST in the multipart/form-data format, including binary file parts. The
>> resulting blobs can then be uploaded with HttpRequest. For example, Gmail
>> uses this when sending/saving messages with attachments that had been
>> queued
>> for delivery while offline.
>>
>>
> Currently, the methods defined on the FileData interface in the FileAPI
> specification do not provide the ability to generate chunks of binary data
> within script, which can then be appended to a FileData object.  Also, we
> need more discussion around storing FileData objects (files) between
> sessions.  Right now we have on the table a URL that refers to files that is
> short lived, and a method that splices a FileData object asynchronously
> (with offset and length).  These haven't been added to the draft that's in
> circulation, but I hope to add them shortly.
>
> I'd like to make sure I understand your use case.
>
> If an attachment is queued already, doesn't Blob.slice(offset, length)
> generate *new* Blobs from that data, which can in turn be used with
> HttpRequest?  It seems that you need:
>
> void append(data) (defined on BlobBuilder)
>
> to extract "data" from storage and then append it to any other Blob, and
> then call getAsBlob() (defined on BlobBuilder) to coin a new Blob, with the
> new data appended.
> For now, can a byte-ranged splice method defined on FileData (to
> asynchronously create "ranged" FileData objects) satisfy your use case?
>  That way, we can discuss file data persistence a bit more, and work on this
> in another edition of the specification.
> I'll report back with a draft.
>
> -- A*
>

Re: File API Feedback

[Michael Nordman]

On Tue, Jul 21, 2009 at 6:20 PM, Arun Ranganathan  wrote:

> Michael Nordman wrote:
>
>> The BlobBuilder.append() method is central to the use-case I'm referring
>> to.
>> builder.append(string data that comprises the multipart/form-data header);
>> builder.append(string data that comprises simple text part);
>> builder.append(string data that comprises the start of a binary part);
>> builder.append(fileBlob); // the file itself
>> builder.append(string data that closes the binary part);
>> // perhaps more binary and text parts
>> builder.append(string data that closes the multipart/form-data);
>> multipartBlob = builder.getAsBlob();
>>
>>
> OK, I understand your use case much better now :)
>
> So, currently, we provide a fileData.getAsText(fileCallback, encoding,
> error)
>
> and
>
> within fileCallback (a callback function) you can get the file's data as a
> string (say a UTF-8 encoded string, or any encoding specified by
> "encoding"); it is passed as an argument to fileCallback.
>
> Then, you can append to the string (or prepend, in the case of the header)
> using the standard JavaScript string methods.  And you can, of course, post
> the string via XHR.
>
> Along with a similarly asynchronous splice method to generate "ranged"
> subsets of a FileData object, I think that we've got *most* of the features
> exposed by both BlobBuilder and Blob.


I don't follow... how would you use the FileData and XHR2 apis to do what I
outlined with blob builder... can you show me in a similar amount of pseudo
code?


>
>
> Have I missed something important?
>
>>
>> Splice doesn't come into play for the use case described above. The
>> motivation for the .splice method is to "chunk" very large files into many
>> parts, and to upload them individually... think large YouTube videos
>>
> Understood.  That is why I am adding it to the specification :-)
>
> -- A*
>
>