Re: [whatwg] Improvement of the Application Cache
Sounds related to "Programmable HTTP Caching and Serving" (formerly titled DataCache API): http://dev.w3.org/2006/webapi/DataCache/ [[[ This document defines APIs for off-line serving of requests to HTTP resources using static and dynamic responses. It extends the function of application caches defined in HTML5. ]]] - Joe On Mar 3, 2011, at 1:50 PM, Michael Nordman wrote: > Sounds like there are at least two feature requests here... > > 1) Some way of not adding the 'master' entry to the cache. This is a common > feature request, there's another thread about it specificially titled > "Application Cache for on-line sites". > > 2) The ability to add(), remove(), and enumerate() individual urls in the > appcache. Long ago, there were interfaces on the appcache drawing board to > allow that. They got removed for a variety of reasons including "to start > simpler". A couple of years later, it may make sense to revisit these kind > of features, although there is another repository also capable of storing > ad-hoc collection of resources now (FileSystem), so i'm not sure this > feature really needs to be in the appcache. > > @Hixie... any idea when the appcache feature set will be up for a growth > spurt? I think there's an appetite for another round of features in the > offline app developers that i communicate with. There's been some recent > interest here in pursuing a means of programatically producing a response > instead of just returning static content. > > > > On Wed, Mar 2, 2011 at 7:40 AM, Edward Gerhold < > edward.gerh...@googlemail.com> wrote: > >> Hello, >> >> i would like to suggest an improvement for the Offline Web applications. >> >> Problem: >> I´ve found out, that i can not Cache my Joomla! Content Management System. >> Of course i´ve read and heard about, that the application cache is for >> static pages. >> But with a little change to the spec and the implementations, it would be >> possible to cache >> more than static pages. >> >> I would like to cache my Joomla! system. To put the scripts, css and images >> into the cache. >> I would like to add the appcache manifest to the index.php file of the >> Joomla Template. >> What happens is, that the index.php is cached once and not updated again. I >> can not view >> new articles. The problem is, that i can neither update the Master File, >> nor >> whitelist it. >> >> And this is, what my request or suggestion is about. I would like to >> whitelist the Master >> file, where the appcache manifest is installed in. Or i would like to >> update >> this file, or any >> file else, i would like to update, on demand. >> >> If there is any possibility, to do that already, please tell me. But i >> think >> that is not the case. >> >> Caching the CMS by making it possible to update or to whitelist certain >> files, the always >> dynamic frontpage or /index.php, would be the hammer to nail the board on >> the storage. >> >> Rules: >> The things, which should be considered are: *To allow to fetch the Master >> file, e.g. index.php* >> *in Joomla! over the NETWORK,* while any other file in the manifest get´s >> fetched or cached like >> before. Which is the most important for me, to get Joomla! into the cache. >> >> Javascript: >> For the script i would like to add *applicationCache.updateMaster()*, which >> forces the browser >> to fetch the file again. I think, this is impossible today, to update >> exactly this file. For the function, >> i could add a button to my page, to let the user choose to update the >> file. >> The second function would be *applicationCache.updateFile(url)*, which >> could >> be triggered by >> a button and script, too. I could let the user update certain articles. >> With that i would like to suggest* applicationCache.addToCache(url)* to add >> files manually or >> programmatic, which can not be determined by the manifest. Urls like new >> articles (*), i would >> like to read offline. I would like to add them to the cache, if the link >> appears, maybe on the >> frontpage. I would have to add the manifest to the CMS anyways, so i could >> add a few >> more functions to the page, of course. * >> applicationCache.removeFromCache(url)* should >> be obvious and helpful with the other functions. >> Good would be, to be able to iterate through the list of cached objects and >> even the manifest, >> with the update, add, remove functions, it would be very useful to work >> with >> the filenames and >> parameters. >> >> [(*) I could let the user decide wether he wants to download my mp3 files >> to >> the appcache or not, >> and fulfill the wish with the javascript functions. Maybe he´s got no bytes >> left or wants only the >> lyrics.] >> >> Conclusion: >> The application cache is very powerful. But it is very disappointing, that >> it is only useful for static >> pages. With a little improvement to the Offline Web applications chapter, >> and of course to the browsers, >> it would be possible to cac
Re: [whatwg] Appcache feedback (various threads)
On Aug 12, 2010, at 3:29 PM, Ian Hickson wrote: >> These quotas are often global, some kind of user setting, or are >> per-origin. Application Caches are missing such a quota. >> >> The entire "Disk Space" section of Web SQL Databases could equally apply >> to Application Caches: http://dev.w3.org/html5/webdatabase/#disk-space > > I suppose that's not unreasonable. I've added a similar section to the > appcache section. Excellent. The text [1][2] looks great. >> The likely behavior would be the user agent emits an "error" event. >> However, storage limits are not specified in the spec as an error >> condition: >> http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#event-appcache-error > > In general the specification does not specify behaviour in handling > constraints of the operating environment (e.g. out of memory, out of disk > space). I've added some generic handling for this. We can add more > detailed error reporting in a future version if there are good reasons to > do this. Good to know. Thanks. - Joe [1]: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#disk-space [2]: http://html5.org/tools/web-apps-tracker?from=5286&to=5287
[whatwg] Application Cache Quota Proposal
Proposal to Add Per-Origin Quotas to Application Cache: --- Application Caches right now do not have any storage limitations. This has clear issues when used on devices with limited storage capacity. Other programmatic client side storages do have quotas mentioned in their specs or in practice: - Web Storage StorageAreas (localStorage and sessionStorage) QUOTA_EXCEEDED_ERR exception when reaching their quota. http://dev.w3.org/html5/webstorage/ - Web SQL Databases QUOTA_EXCEEDED_ERR exception when reaching its quota. http://dev.w3.org/html5/webdatabase/ - Indexed DB RECOVERABLE_ERR exception when "... or the storage quota was reached ... " http://www.w3.org/TR/IndexedDB/ - Cookies "Minimum size" a client should support was stated in the HTTP Spec as 4kb, so that is often quoted and respected as the size to not exceed. These quotas are often global, some kind of user setting, or are per-origin. Application Caches are missing such a quota. The entire "Disk Space" section of Web SQL Databases could equally apply to Application Caches: http://dev.w3.org/html5/webdatabase/#disk-space Current Behavior: - The manifest file specifies the resources to cache, and the user agent caches each resource. There may be multiple application caches for a single domain. Be it through multiple Application Hosts, or multiple Application Groups (unique by their manifest URL, but not domain). A user agent can do as it pleases, but the only real limit is the remaining space of the file system. If a resource cannot be cached because there is no more space left, there is no way around that. The likely behavior would be the user agent emits an "error" event. However, storage limits are not specified in the spec as an error condition: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#event-appcache-error Suggested Additions to the Spec: Under the "error" event description [1], suggest that it could be fired as a result of reaching a quota limit. The spec often uses the phrase "store this resource". That part could just be described somewhere in the specification as possibly raising an "error" event. The phrasing in the other Specs are: [[ (Setting could fail if, e.g., the user has disabled storage for the site, or if the quota has been exceeded.) ]] [[ The statement failed because there was not enough remaining storage space, or the storage quota was reached and the user declined to give more space to the database. ]] [[ For example, there was not enough remaining storage space, or the storage quota was reached and the user declined to give more space to the database. ]] I would most like to see something like the "Disk Space" section of Web SQL Databases. This section describes actions the user agent "should" take to give the user some control over the quota. Such as viewing, setting, and being notified when the quota is reached: http://dev.w3.org/html5/webdatabase/#disk-space Use Cases: -- These would be the same as the use cases for quotas or storage limits on all the other client side storage implementations which have quotas. But this would also be useful to define the expected behavior when reaching such limits for interoperability between Application Cache implementations. Cheers, - Joe [1]: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#event-appcache-error
Re: [whatwg] Should window.name be [Replaceable]?
On Jul 4, 2010, at 9:35 PM, Biju wrote: > On Mon, Jul 5, 2010 at 12:05 AM, Joseph Pecoraro wrote: >>- I didn't have access to IE to test. I'd appreciate >> someone giving it a shot to see how they act. > > On IE8 it is printing "string,string,string" Thanks for checking! So its not Replaceable. > And it looks like in Firefox > 1) name = with_a_number changes type to matched the assigned value > 2) window.name = with_a_number keeps the previous type Yes. So there is no way to access the real window.name once it has been "replaced". - Joe
[whatwg] Should window.name be [Replaceable]?
Should window.name be [Replaceable]? There are a number of "[Replaceable]" [1] properties on the window object. However, "window.name" is not marked as such [2]. Not being marked as replaceable means that if authors use a global variable named "name" it will be coerced into a string. For many this causes confusion. It seems some browsers have opted to make it replaceable. Test case. You can test your browser at [3]. (function() { var arr = []; arr.push( typeof window.name ); window.name = 1; arr.push( typeof window.name ); name = 1; // "number" if [Replaceable], "string" otherwise arr.push( typeof window.name ); alert( arr ); })(); Browser Results: - Safari 5 / WebKit Nightly and Chrome 5 NOT [Replaceable], printing "string,string,string" - Firefox 3.6.3 / 3.7a5 and Opera 10.5.3 / 10.6 [Replaceable], printing "string,string,number" - I didn't have access to IE to test. I'd appreciate someone giving it a shot to see how they act. Has there been discussion on this in the past? I searched and didn't find any discussion. - Joe [1]: http://dev.w3.org/2006/webapi/WebIDL/#Replaceable [2]: http://www.whatwg.org/specs/web-apps/current-work/multipage/browsers.html#the-window-object [3]: https://bug-19967-attachments.webkit.org/attachment.cgi?id=60491
Re: [whatwg] Form Validity Ambiguity - patternMismatch
On Apr 20, 2010, at 6:16 AM, Mounir Lamouri wrote: > Actually, if you are looking for something more normative (the table is > non-normative), at the end of each input element states description, you > can found the list of attributes which apply/don't apply. AFAIK, the > attributes/methods never mention in which states they apply. I suppose > it is to prevent mentioning things twice thus doubling the needed > changes and the chances of incoherence/mistakes. You're correct! Thanks for pointing out that section. I still think this could be clarified. I feel that a majority of the time someone is going to read patternMismatch and say "oh, I just need a pattern attribute" and not think to check that the pattern attribute only applies to particular types. Does anyone else agree or should this be left alone? - Joe
[whatwg] Form Validity Ambiguity - patternMismatch
The section describing the pattern attribute does not explain that it only applies to particular input types [1]. However, one of the tables showing which content attributes apply to which input types shows the pattern attribute only applies to particular input types. Namely text, search, url, telephone, email, and password but not dates, times, numbers, hidden and others: This disconnect could lead to ambiguous implementations of ValidityState's patternMismatch which is simply described in multiple places as: [[ When a control has a value that doesn't satisfy the pattern attribute. ]] [3] [[ returns true if the element's value doesn't match the provided pattern; false otherwise. ]] [4] These two definitions of patternMismatch do not mention that they only apply on input type's that the pattern attribute applies to. If that is the case then I recommend clarifying that constraint in the description of the pattern attribute [1] and when describing patternMismatch [3][4]. If that is not that the case then I don't see the purpose of the table showing that the "pattern" attribute only applies to particular input types, and I would like clarification there (like the row being removed or all checkboxes?) [2]. This may also apply to other validity states as well, I haven't checked others yet. Cheers! - Joe [1]: http://www.whatwg.org/specs/web-apps/current-work/multipage/common-input-element-attributes.html#attr-input-pattern [2]: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-input-element.html#the-input-element [3]: http://www.whatwg.org/specs/web-apps/current-work/multipage/association-of-controls-and-forms.html#suffering-from-a-pattern-mismatch [4]: http://www.whatwg.org/specs/web-apps/current-work/multipage/association-of-controls-and-forms.html#the-constraint-validation-api
Re: [whatwg] Appcache feedback
> On Thu, 17 Dec 2009 22:44:01 +0100, Ian Hickson wrote: >> We could delay the application cache download process so that it doesn't >> start until after the 'load' event has fired. Does anyone have an opinion >> on this? On Dec 17, 2009, at 5: 24PM, Michael Nordman wrote: > I don't think we'd have to delay the update job, just the delivery of any > events associated with the appcache until 'load' has happened to get the > desired effect. On Dec 18, 2009, at 7: 51AM, Anne van Kesteren wrote: > I believe we (Opera) found this is what at least some implementations > are doing now and we do this as well. What Michael suggested makes sense > and I suppose as the effects are indistinguishable in the end that > (delaying either the events or the download process) should be fine. Mike's suggestion sounds good. Is this in the queue to be added or could this have been forgotten about? There was a change around the time this email thread occurred that clarified that the download process should be performed in the background, but nothing about the events. - Joseph Pecoraro
Re: [whatwg] Appcache feedback
On Dec 17, 2009, at 4: 44PM, Ian Hickson wrote: > Another conforming sequence of events would be: > > 1. The parser's first parsing task begins. > 2. As soon as the manifest="" attribute is parsed, the application cache > download process begins. It queues a task to dispatch the 'checking' > event. > 3. The parser's first parsing task ends. > 4. The event loop spins, and runs the next task, which is the 'checking' > event. No scripts have yet run, so no handlers are registered. Nothing > happens. > 5. The parser's second parsing task begins. It will parse the script, etc. > > [snip moved below..] > > We could delay the application cache download process so that it doesn't > start until after the 'load' event has fired. Does anyone have an opinion > on this? From an application developer standpoint I think that would be very nice. I cannot comment on a this from an implementors perspective (yet). It seems important considering the following text from the Spec: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#downloading-or-updating-an-application-cache [[ Certain events fired during the application cache download process allow the script to override the display of such an interface. The goal of this is to allow Web applications to provide more seamless update mechanisms, hiding from the user the mechanics of the application cache mechanism. User agents may display user interfaces independent of this, but are encouraged to not show prominent update progress notifications for applications that cancel the relevant events. ]] It seems pointless to provide hooks in the API that allow for a "custom interface", when fundamentally they may never be triggered in an otherwise compliant user agent. > You can work around all this by checking the .status attribute when you > first hook up the event listeners. This is even worse. To me, this means the application developer cannot rely on certain (any?) events, so he/she would need to build a completely new API on top? Developers will still likely be caught searching for bugs in their own code (like I did) wondering why behavior is different between browsers. Thanks for the quick response, - Joe
[whatwg] [ApplicationCache] Clarify When Events Fire
I'd like some clarification on exactly when the ApplicationCache events should fire. Specifically the events that are likely to fire early, such as the "checking" or "downloading" events. In "6.9.4 Downloading or updating an application cache", the specification currently says the following for the "checking" event (step 4): http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html#dfnReturnLink-0 [[ If these steps were invoked with a cache host, and the status of cache group is checking or downloading, then queue a task to fire a simple event named checking that is cancelable at the ApplicationCache singleton of that cache host. ... ]] What should the proper behavior be in the following scenario: Application Cache Bug window.applicationCache.onchecking = function() { alert('checking'); } You should get an alert on reloads! Firefox triggers the alert. WebKit does not trigger the alert. There is an open WebKit bug on this topic [1] which describes the thought process behind WebKit's behavior. Taken from the comments: 1) As soon as body manifest attribute is processed, application cache update begins, and a zero-timer task to dispatch a checking event is posted for later execution. 2)
Re: [whatwg] Reliably Minimize Reflows
// Single reflow would be triggered at the end of batchUpdate document.batchUpdate(function() { ... }); As a UA developer, I'd not be all that happy implementing this, since you can stay inside the batchUpdate more or less forever (e.g. by putting up alerts or doing sync XHR). In any case, it seems unnecessary. I was aware of the potential for a thread to sit inside a batchUpdate. I guess the basic idea would have been that batchUpdate would be a hint to the UA. However, I can't think of any other situations where there are API functions that are "hints" so maybe this isn't appropriate for a spec? "Browsers may choose to wait until the end of a script thread before reflowing to show the changes. Opera will wait until enough changes have been made, enough time has elapsed, or the end of the thread is reached. This means that if the changes happen quickly enough in the same thread, they may only produce one reflow. However, this cannot be relied on, especially considering the various different speeds of devices that Opera runs on." Note that Gecko will not perform layout while your script is running, unless you ask for layout information. If Opera has a different behavior, I assume they have a good reason for it, right? Why are you sure you want to override that reason? I don't think of this as overriding any reasons/behaviors. I thought that this would do more to justify those reasons/behaviors. - Joe
[whatwg] Reliably Minimize Reflows
A task that developers are often faced with is applying many style updates to multiple Nodes in the DOM. An example would be showing/ hiding all divs with the "foo" style class: // Hide multiple divs var foos = document.getElementsByClassName('foo'); for (var i=0, len=foos.length; i < len; ++i) foos[i].style.display = 'none'; As I understand it, if there are 20 .foo elements then updating the style on each _could_ cause 20 individual reflows. It would be best if the developer can narrow that down to a single reflow. Minimizing reflows is a well known suggestion for speeding up web pages [1][2]. The most natural and generalized API that I can think of would be something like: // Single reflow would be triggered at the end of batchUpdate document.batchUpdate(function() { ... }); Opera mentions a little about its reflow algorithms at [2] with raises some interesting points. Here is the relevant portion: "Browsers may choose to wait until the end of a script thread before reflowing to show the changes. Opera will wait until enough changes have been made, enough time has elapsed, or the end of the thread is reached. This means that if the changes happen quickly enough in the same thread, they may only produce one reflow. However, this cannot be relied on, especially considering the various different speeds of devices that Opera runs on." The idea here is that "batchUpdate" would be a reliable way to minimize reflows. Does this sound like a worthwhile improvement? Cheers, Joseph Pecoraro [1]: http://code.google.com/speed/articles/reflow.html [2]: http://dev.opera.com/articles/view/efficient-javascript/?page=3
Re: [whatwg] document.head
On Sep 20, 2009, at 10: 29PM, Garrett Smith wrote: On Sun, 20 Sep 2009 16:15:11 -0400, Joseph Pecoraro > Something like this almost always works: var head = document.documentElement.firstChild The documentElement.firstChild cannot be expected to be head. It could be a text node. For example:- ... the first child node of HTML looks like a textNode with the value "\n\n\u0020\u0020". document.getElementsByTagName("head")[0] could be expected to produce (in a valid HTML document) a result that is more consistent than document.firstChild. Even despite the white space, I got the element in Safari, Firefox, and Opera. But, you're absolutely right, I'm not saying my suggestion is bulletproof, only that it can be expected to perform far better! However, I am interested in seeing a solution that is bulletproof and one that performs well! Namely, "document.head". Cheers, - Joe
Re: [whatwg] document.head
On Sep 20, 2009, at 4: 24PM, Juriy Zaytsev wrote: Speaking of `document.head`, I think Mootools does exactly that. Good thinking. I took a look at some JavaScript Libraries / Frameworks. Here are some quick counts of how many times I could see that they use the "getElementsByTagName" method of getting the "head" element: - MooTools 1.2.3 has: 2 - jQuery 1.3.2 has: 2 - Ext 3.0.0 has: 5 - YUI 3.0.0b1 has: >10 - Prototype 1.6.1, MochiKit 1.4.2, Dojo 1.3.2 have: 0 I am not making any criticism of the libraries. I'm just re-enforcing the fact that it is a popular method of getting the "head" element, and adding document.head would be a worthwhile improvement. - Joe
Re: [whatwg] document.head
On Sep 20, 2009, at 3:57 PM, Michael A. Puls II wrote: I think it'd be cool to have to complement document.documentElement and document.body. On Sep 20, 2009, at 4: 00PM, Juriy Zaytsev wrote: Surely better than abominable – `document.getElementsByTagName ('head')[0]` :) I agree. Unfortunately that is the most popular method I've seen. There are better solutions, but they haven't caught on. I think a smarter idea would be to look at the children of the element. Something like this almost always works: var head = document.documentElement.firstChild - Joe
[whatwg] document.head
Was there any discussion for including "document.head" in HTML5? Searching the mailing list shows document.head show up a few times in example code [1][2]. However, there has been no proposal, and it is not mentioned in the document's IDL [3] in the Spec. Developers often do the following to get a reference to the element (top 2 google results) [4]: var head = document.getElementsByTagName('head')[0]; Its useful for developers to use that reference to manipulate/append stylesheets, scripts, and
Re: [whatwg] Access the Response Headers for the Current Document
On Jul 28, 2009, at 9: 21PM, Ian Hickson wrote: Use Cases: Any that apply to XHR accessing their response headers would certainly apply here. Some thoughts are accessing the Content-Type header or Custom Headers and acting accordingly. You can just include the data straight into the page, for now. It's really clear what the use cases would actually be in practice. True, but that feels like a hack. If the HTTP protocol contains the data you need, then a server-side script may try to provide the data and may possibly provide an incorrect value. Likewise at the very least its a duplication of data being sent. This is certainly better then the current method, but not optimal. Come up with a clear description of the problem that needs to be solved: Cannot access the Response Headers for the current document in Javascript. Any there Browser Implementors out there that agree with this? If so, any thoughts on the best ways to expose the current page's request headers to Javascript? Certainly they are readonly, modifying them seems to be useless. How about keeping consistent with the XHR interface with something like: document.getAllResponseHeaders() and document.getResponseHeader(header) This is something that might make sense for a future version, but in the absence of a compelling need for this, I'm going to skip adding this in this version. I originally helped someone in an IRC channel with this question. He wanted to check a "Date" header being sent from his server, via Javascript. I don't know what his exact reason was. We provided him the same solutions mentioned here. However, like Adam de Boor suggested, a use case could be detecting proxies. The use case that I thought of was using custom headers to ensure requests go to a certain server in a cluster, perhaps to maintain a session with a reasonable cache. But that isn't really compelling and probably not very common.
Re: [whatwg] A New Way Forward for HTML5
I think we need an approach that doesn't involve in-flow links... I'm just not sure what the right solution is. Maybe alt-double-clicking should show a menu with two options, "submit comment here" or "change section status"? Alt-Double Click doesn't sound very discoverable. Even if I knew that shortcut I'd probably forget at some point. Maybe having something position:fixed would be better because there is something visible and reachable at all times. The problem then would be automatically determining which section is being commented on. Its certainly not as straight-forward as clicking on the section. Just some ideas. I'm very interested in seeing a commenting system. - Joe
[whatwg] Access the Response Headers for the Current Document
It seems like an oversight that Javascript can read response headers off of XHR but not for the current document. So in order to find out the headers for the current document you would need to make another request, refetching the current page, to find that out [1]. Use Cases: Any that apply to XHR accessing their response headers would certainly apply here. Some thoughts are accessing the Content-Type header or Custom Headers and acting accordingly. Come up with a clear description of the problem that needs to be solved: Cannot access the Response Headers for the current document in Javascript. Any there Browser Implementors out there that agree with this? If so, any thoughts on the best ways to expose the current page's request headers to Javascript? Certainly they are readonly, modifying them seems to be useless. How about keeping consistent with the XHR interface with something like: document.getAllResponseHeaders() and document.getResponseHeader(header) Cheers, Joseph Pecoraro [1] Example: http://bogojoker.com/x/xhr/headers.html
Re: [whatwg] Browser Bundled Javascript Repository
On Jul 13, 2009, at 3: 01PM, Aryeh Gregor wrote: How about you have an extra HTTP header like "X-Content-Hash"? This could provide a SHA256 hash (or something else that looks safe for now, progressively upgradeable) of the content. The browser can keep its cached copies of these files indexed by hash. If it tries downloading a file, and notices that the hash is the same as a file already downloaded, it can terminate the HTTP connection and use the existing file (even if it's from a different site). It will then proceed as though it had actually downloaded the file: e.g., it will respect the Expires headers separately (two sites might serve the same file but have different expectations about how likely it is to change). I think thats brilliant. Its a cache that works across all sites. On Jul 13, 2009, at 3: 01PM, Aryeh Gregor wrote: The most obvious place to solve this seems to be HTTP, not HTML. HTTP is closer to the resource itself. If you do something with HTML, like an extra attribute, then you're going to get authors updating the HTML but not the thing it points to or vice versa. An ETag-like solution would be implemented either in the web server or whatever script is serving the content, and those should always know whether the file has changed. (Modulo pathological behavior like something changing the file and then forging the mtime/ctime.) I agree. On Jul 13, 2009, at 4: 20PM, Aryeh Gregor wrote: Does anyone have statistics on how useful this would be in real life? I suspect only marginally. I think this is the most important aspect of this idea. We don't yet know if this is worth doing yet. It is likely only preventing the initial download of some files. However some of the initial framework sizes are getting hefty: (These were the sizes just pulled from google's hosted libraries for the latest versions) 91K dojo.xd.js 79K ext-core.js 182K jquery-ui.min.js 56K jquery.min.js 127K prototype.js 2.6K scriptaculous.js 10K swfobject.js 27K yuiloader-min.js I'm guessing that mobile browsers would benefit from not needing to download a few of those 100KB downloads and use up an HTTP Connection to do so. And speaking of mobile, there are some mobile specific web application frameworks (I'm thinking of ones for iPhone web apps) that don't have a single cache point (like google) that weigh in pretty heavily. Real statistics would make it obvious wether or not this is a good idea. But even still, I like the HTTP idea, because at that point its just a more efficient way to cache files, by content, across all the entire web, rather then site specific and by name/URL. - Joe
Re: [whatwg] Browser Bundled Javascript Repository
But linking external scripts does have a problem in that you have to trust the site you're linking not to change the script (or get compromised) to add malicious features. A cryptographic hash of the file you expect could be used to mitigate this issue, perhaps for other types of file too. And such a feature could fall within HTML5's purview. For example:
Re: [whatwg] Storage Events for a Specific Storage Area
We could make Storage into an EventTarget and also fire events on there, sure. As Jeremy said, this is something that might best be done in the next version; we're still trying to get the current features implemented in a stable and consistent fashion as it is. Sounds great. Although wouldn't now be the best time to introduce something new? Its not a breaking change (theoretically), its an additional feature. Its my understanding that most of Web Storage is undergoing changes (B-trees and SQL orthogonally), but the Storage interface is not going to change much? Is that because of current browser support? In general, though, I don't think it's critical, since you can easily work around it by checking which was changed based on the event data Fair enough. =) (I agree that your proposal would have made more sense. I'm sorry my learning experiences have such a detrimental effect on the Web! :-) ) We're all learning. =P -- Ian Hickson U+1047E) \._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _ \ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'-- (,_..'`-.;.' - Joe
Re: [whatwg] Browser Bundled Javascript Repository
Some quick feedback before I heed Ian's advice and try to formalize things a little more. As a browser vendor (or at least someone working for one, specifically on script-loading), I would definitely be interested in something like this. However there are some hard problems to be solved: I looked through the mailing list and I take it you're from Mozilla. Correct me if I'm wrong. =) 1. How do we choose which libraries to pre-package? I'd rather not be in charge of ruling which libraries are popular enough or cool enough to warrant inclusion. It'd be very nice if the "library cache" were populated dynamically as the user used the web. Yes this point did come up. I believe that the cache can be grown dynamically, but I don't think that the cache could be reliably used without some indication or opt-in from the html (and thus the developer or some automated process). Files can be compared with each other very quickly, and SHA hashes even quicker, to determine that indeed the same content is being sent from multiple URLs, thus allowing potential for the cache to grow dynamically with frequently encountered content. But, again, I think the problem is then on the developer to opt-in to the service, otherwise the cache is just a speculation. 2. How do we deal with identifying libraries. As Aaron Boodman pointed out, SHA hashes means that you can't make upgrades for security problems etc. I still don't see this as an issue. As long as it is a SHA hash of the content, then its the content that matters. If a developer is using a script with a security problem then they should change their script. At this point a browser could go the extra mile and "attempt" to fix it for them by updating the script but thats another "speculation". The idea of "canonical" names could run into problems though. 3. Compat when the browser doesn't have a library cached. A solution that includes using a different uri for browsers that do support caching than browsers that don't is scary since there is a big risk that the author will forget to update one but not the other. Yes, I do see this as a problem. I gave a response to this before, but I do still see this as being a potential headache point. Tools can easily solve this but thats not a perfect solution. I haven't come up with a good solution to this, but I do have some ideas. I believe all these problems are solvable, but they do need to be solved before considering inclusion in any spec. I'm also not convinced this needs to be included in the HTML5 spec, but that might depend on what the ultimate solution looks like. I agree again. I concluded this feature makes the most sense for a restricted environment, such as developing mobile web apps. Those developers are already faced with some different conditions, and I think iPhone web development even has some custom syntax. A feature like this could make a much bigger impact in that kind of environment where it could gain traction. / Jonas Thanks for the feedback Jonas. - Joe
Re: [whatwg] Browser Bundled Javascript Repository
I would recommend approaching the browser vendors directly and seeing if they would be interested in implementing this idea, as discussed in this FAQ entry on introducing new features: http://wiki.whatwg.org/wiki/FAQ#Is_there_a_process_for_adding_new_features_to_the_spec.3F I see, thanks Ian. - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
This doesn't break anything in the current spec. So it wouldn't break any existing implementations. I'm also guessing that the groundwork for implementing a feature like this is already in place due to the ubiquity of addEventListener. To be frank, it seems like a lot of bloat though to avoid a simple comparison. Its my thinking that the simple comparison shouldn't be necessary to begin with. Thats why I suggested this. This is a probably a bad analogy, but would it make sense to put a "click" function on just the window object and check every time it fires to see if it fired in the you were interested in? Imagine every single "click" listener you register fires every time you click on something... that doesn't scale. (The reason this is a bad is because these "click" events "bubble". Maybe bubbling is appropriate here)? - Less Listener Functions Fired - Instead of every registered listener getting fired on every "storage" event, only those applicable will be fired. This may mean overall less listeners getting fired, and code that doesn't have to continually check the affected storageArea, leading to potential performance improvements. Actually, assuming only one of the two solutions would exist the same amount of events would fire. A change to localStorage causes an event to be dispatched and likewise a change to sessionStorage causes an event to be dispatched. Having said that, with your solutions more events will fire since the legacy event will have to be dispatched too. What do you mean when you say "more events will fire?" Why not just fire the same "storage" event but now put the if statement that the web developer had to "always write" and put it into the browser's dispatching code. This may be my inexperience with the implementations, but I honestly don't see how this could end up firing more events. I think looking at it this way ("more events") is looking at it sideways and maybe a bit pessimistically. If my first thought is "this is going to be really complex" then my next thought is "how can I simplify this" and if I can't find a way to simplify it I would state why it can't or shouldn't be done. I don't see that here, I just see the "this is really complex" part. I'm guessing that there is a simpler way to look at this, and if not I'd be interested to know why this seems so complex. - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
Jeremy Orlow: Is it too late? It seems as though Joseph's suggestion could be in addition to what's already in the spec. This doesn't break anything in the current spec. So it wouldn't break any existing implementations. I'm also guessing that the groundwork for implementing a feature like this is already in place due to the ubiquity of addEventListener. Pros: - Less Developer Confusion: When a web developer is focusing on a particular Storage area and set a "storage" listener, unless they check the storageArea they may not realize their listener is getting fired for scripts dealing with a different Storage area. I'm not sure if this can be considered a "security" threat but it would certainly limit the potential for problems. - Less Listener Functions Fired - Instead of every registered listener getting fired on every "storage" event, only those applicable will be fired. This may mean overall less listeners getting fired, and code that doesn't have to continually check the affected storageArea, leading to potential performance improvements. Cons: - This is similar to implementing the full EventTarget Interface on Storage areas. However, the only event of importance on a Storage area is "storage" not "click" or "hover" etc. Invalid event types should probably throw an error or something... or maybe this is more in the scope of DOM Events. Anne: By the way, change requests for this specification should be made to public-weba...@w3.org. localStorage and friends are no longer part of HTML5. I read from your (Anne's) blog that Web Storage has split off of HTML5, and I see it on the WebApps homepage [1] but the document itself said to send feedback to either mailing list or the bugzilla [2]. I just subscribed. In the future I'll use the other list for whats on the WebApps homepage. [1]: http://www.w3.org/2008/webapps/charter/ [2]: http://dev.w3.org/html5/webstorage/#status-of-this-document - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
Seems like a reasonable idea. I don't know if it's been brought up before. My guess is that Ian will say this should be re-examined for HTML6, though. Very cool. Let me know if I can help at all to push this idea forward. - Joe
Re: [whatwg] Storage Events for a Specific Storage Area
It sounds like there wasn't any discussion on this. I recently heard talk of other potential Storage areas [2]. That would make this idea even more appealing to me. Does this sound like something worth adding? Any comments? [2]: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020485.html On Jun 17, 2009, at 1: 44PM, Joseph Pecoraro wrote: The "storage" event [1] fires for both sessionStorage and localStorage. To me, this means if you only want to interact with localStorage you will have to manually ensure that it is the storage area being modified: window.addEventListener('storage', function(e) { if ( e.storageArea === localStorage ) { // ... } } Was there any discussion about creating events specific to the storage object, or should that already be possible? I've been playing around with WebKit's Storage implementation, and the following (understandably) is not possible: > localStorage.addEventListener undefined Is there any way to listen to events for a single specific storage area or is the previously mentioned approach preferred? Cheers, Joe [1]: http://dev.w3.org/html5/webstorage/#the-storage-event
[whatwg] Storage Events for a Specific Storage Area
The "storage" event [1] fires for both sessionStorage and localStorage. To me, this means if you only want to interact with localStorage you will have to manually ensure that it is the storage area being modified: window.addEventListener('storage', function(e) { if ( e.storageArea === localStorage ) { // ... } } Was there any discussion about creating events specific to the storage object, or should that already be possible? I've been playing around with WebKit's Storage implementation, and the following (understandably) is not possible: > localStorage.addEventListener undefined Is there any way to listen to events for a single specific storage area or is the previously mentioned approach preferred? Cheers, Joe [1]: http://dev.w3.org/html5/webstorage/#the-storage-event
Re: [whatwg] Browser Bundled Javascript Repository
In the event of a collision there would be huge issues - imagine running someone else's script in your application. Basically XSS - someone could take over your app, steal passwords, do bank transactions on your behalf, etc. Collisions are made easier in plain text than in certs given that your input is not constrained. Aaron Boodman: I think the idea was for browser vendors to select and include these libraries in the browser. So there isn't an obvious (to me) way for an attacker to use hash collisions to create an XSS. Yes, thanks for clearing that up. That said, I don't think content hashes are the right identifier. Using a sha-1 of a specific jquery version would prevent anyone from ever fixing critical bugs in it. There's be all this legacy content out there referring to an outdated version. Assuming there is a buggy version of a JS library: 1. It probably shouldn't be used. 2. The browser vendor can (and should) eliminate it from their repository and proceed with the usual fallback of downloading the script from the
Re: [whatwg] Browser Bundled Javascript Repository
c) fun things would happen with a SHA collision! ;) c) Hehe, I think I detect a hint of sarcasm. If there is a SHA1 collision then you'd probably make a lot of money! C is a serious concern. SHA-1 collisions are now 2^51 - http://eprint.iacr.org/2009/259.pdf This time I didn't detect sarcasm =) I was actually aware of that paper. I saw it on Reddit this past week, and although they complained about the fact that it has not yet been reviewed I think it could very well be valid. Its been known that SHA1 has been theoretically broken (not perfect 2**80) for some time now: (2005) http://www.schneier.com/blog/archives/2005/02/sha1_broken.html However, its application in this Repository idea is not to be a cryptographically secure hash, it would just be to perform a quick, reliable, hash of the contents and to produce a unique identifier. There would be no security concerns in the impossibly rare chance that two scripts hashes collide. Just add some whitespace to the text somewhere! It would even be easy to debug when with standard tools such as Firefox's Firebug and Webkit's Web Inspector. Hahaha =) Also, Git and Mercurial (distributed version control systems) have been using SHA1 for the exact same purpose for years. I'm more familiar with Git's use of SHA1 and it uses it everywhere in the internals (file contents, directory listings, commit history). Finally, if anyone here is seriously concerned with SHA1 just move to SHA-256 or SHA-512. With a repository unlikely to grow into the thousands, much less the millions, the chances of a collision even in 2**51 (2251799813685248 base 10) is bold thinking ;) I'm not attacking anyone here, I'm just clarifying why I think SHA1 is not a bad choice. Collision will always be an issue when a infinite number of things gets reduced to a finite set of values, but the concern negligible when done right. Cheers - Joe
Re: [whatwg] Browser Bundled Javascript Repository
On Mon, Jun 15, 2009 at 1:09 PM, Joseph Pecoraro wrote: Dion: The problem here is that isn't backwards compatible and thus no-one will really be able to use it. I thought the original idea was backwards compatible. Maybe not the URN Schemes. If the original idea is not, could you point out the issues? The URN schemes isn't compatible. The SHA hash idea is do-able, but as Oliver pointed out is impractical: a) devs will forget to update it, b) looks ugly, c) fun things would happen with a SHA collision! ;) a) Solved by Validation - I can't think of anything much better then that. =( b) Canonical Listing - This shouldn't be too difficult to distribute from a central source or some convention. c) Hehe, I think I detect a hint of sarcasm. If there is a SHA1 collision then you'd probably make a lot of money! Dion: You then also get into the "how do I get my library into the browser?" Enough widespread usage of a library is a clear indicator for adoption into a browser bundle. Dynamically growing repositories could optimize per computer for the particular user's browsing habits (assuming developers would mark their scripts with the identifiers). You can have the same problem with what libraries will Google include in its CDN. Although it may be easier for Google to host just about any library if it already has a CDN setup. This was a real problem for us. How much is "enough" ? We started to get inundated with requests for people to put libraries up there. Lets the browsers decide. And I can't make any reasonable suggestions without getting real world data, something I haven't tried to do yet. But yes, this is a good point, something that is extremely flexible / variable. Dion: After mulling this over with the Google CDN work, I think that using HTTP and the browser mechanisms that we have now gives us a lot without any of these issues. I was afraid of this. This is a completely valid point. I guess it sounds like too much work for too little gain? I don't want to stop you from working on these ideas. The core problem that we tend to download the same crap all the time is real, and I look forward to seeing people come up with interesting solutions. Thanks for the support. My thoughts are beginning to look like this: - Javascript Frameworks are downloaded all the time on many domains. This is a special case. - Those who benefit the most are the ones that can't space the extra request or large caches. This makes me think mobile browsers would get the biggest benefit. - I think the iPhone had some special html syntax for its mobile webpages, maybe they can sneak this in if it proves useful to them. - Joe
Re: [whatwg] Browser Bundled Javascript Repository
Chris Holland: But you're right, this is all a lot of end-user intervention: it would be a slightly, err, very painful process of installing a browser plugin, which is currently very-much of a user opt-in process, and not something very practical. [...]. I'm just trying to find ways to leverage a lot of what's already there. Yes, I don't like the idea of requiring users to act. But I see what you were tying to do with reuse. Interesting thing is the same scheme could be leveraged for local CSS extensions: I thought about this as well, but CSS is far less likely to be duplicated across multiple sites. There are plenty of CSS Frameworks but I feel that none have picked up enough dominance for this kind of optimization to be useful. You do mention what looks like urn schemes and extending this idea to CSS. I was specifically thinking of javascript because of its widespread use of libraries/frameworks. Using URN Schemes could let this repository idea extend to more then just javascript, however I don't think any other type of resource (CSS, Images, Etc.) have this unique pattern of "the exact same content being served on thousands of different domains." To handle users who don't have the "ibdom" javascript extension installed, developers could add something like this to their document: (assuming a decent library which declares a top-level object/namespace): if (!window.IBDOM) { var newScript = document.createElement("script"); scr.setAttribute("type","text/javascript"); scr.setAttribute("src","/path/to/ibdom.0.2.js"); document.appendChild(newScript); } Although the idea is the same (have a fall back plan if a repository lookup is ignored or fails), I think this is needlessly complex compared to just adding a new attribute on the
Re: [whatwg] Browser Bundled Javascript Repository
Dion: The problem here is that isn't backwards compatible and thus no-one will really be able to use it. I thought the original idea was backwards compatible. Maybe not the URN Schemes. If the original idea is not, could you point out the issues? Dion: You then also get into the "how do I get my library into the browser?" Enough widespread usage of a library is a clear indicator for adoption into a browser bundle. Dynamically growing repositories could optimize per computer for the particular user's browsing habits (assuming developers would mark their scripts with the identifiers). You can have the same problem with what libraries will Google include in its CDN. Although it may be easier for Google to host just about any library if it already has a CDN setup. Dion: After mulling this over with the Google CDN work, I think that using HTTP and the browser mechanisms that we have now gives us a lot without any of these issues. I was afraid of this. This is a completely valid point. I guess it sounds like too much work for too little gain? - Joe
Re: [whatwg] Browser Bundled Javascript Repository
Library vendors would then ship their releases as browser plugins, using existing discovery mechanisms, as well as software update mechanisms. -chris This sounds to me as though the user would have to download a browser plugin. I would hope this would be as transparent as possible to the user. Maybe I'm misunderstanding the discovery mechanisms you're talking about. Could you expand on this? I do think that a URI prefix is a neat idea. This could eliminate the need for a new attribute. Is it as backwards-compatible? - Joe
Re: [whatwg] Browser Bundled Javascript Repository
The common JavaScript libraries should be identified using urn scheme with JavaScript namespace, as in Chris Chris, this was what I was originally thinking of when I read about tag-uri's in the Atom Publishing Protocol (which I think are now defunct). URN Schemes: http://www.w3.org/TR/uri-clarification/ I don't know too much about URN Scheme's but I don't think they provide as much benefit as a hash would be (ugly as they are). In the original email I alluded to canonical or easy to use identifiers that would be alias's for the hash: Maybe a URN Scheme would be optimal for a canonical name, but I still like the SHA1 hash for verification/validation and dynamic growth. - Joe
Re: [whatwg] Browser Bundled Javascript Repository
Pros: - Pre-Compiled: By bundling known JS Libraries with the browser, the browser could store a more efficient representation of the file. For instance pre-compiled into Bytecode or something else browser specific. I think something needs to be clarified wrt to compile times and the like. In the WebKit project we do a large amount of performance analysis and except in the most trivial of cases compile time just doesn't show up as being remotely significant in any profiles. Additionally the way JS works, certain forms of static analysis result in behaviour that cannot reasonably be cached. Finally the optimised object lookup and function call behaviour employed by JavaScriptCore, V8 and (i *think*) TraceMonkey is not amenable to caching, even within a single browser session, so for modern engines i do not believe caching bytecode or native is really reasonable -- i suspect the logic required to make this safe would not be significantly cheaper than just compiling anyway. I noticed this came up on the WebKit mailing list recently and they said the same thing as you, that compile time was insignificant. Thanks for expanding on this: https://lists.webkit.org/pipermail/webkit-dev/2009-May/007657.html Although this response sounded slightly more promising: https://lists.webkit.org/pipermail/webkit-dev/2009-May/007682.html - Less HTTP Requests / Cache Checks: If a library is in the repository no request is needed. Cache checks don't need to be performed. Also, for the 100 sites you visit that all send you the equivalent jquery.js you now would send 0 requests. I think this would be enticing to mobile browsers which would benefit from this Space vs. Time tradeoff. I believe http can specify how long you should wait before validating the cached copy of a resource so i'm not know if this is a real win, but i'm not a networking person so am not entirely sure of this :D I believe you're correct. HTTP E-Tags and Expires headers can influence browsers to cache resources for very long times. Unfortunately I don't think that a lot of developers take advantage of these tags (this is a bad argument but worth mentioning). Although Yahoo's Y-Slow and Google's Page-Speed extensions have opened many developers eyes to ways they can improve their site's performance. The real gain however, would be if you visited 100 different websites that all needed the same script and you wouldn't have to make a single request, or cache a single resource, due to using the script in the repository. I think this sounds better. - Standardizing Identifier For Libraries: Providing a common identifier for libraries would be open for discussion. The best idea I've had would be to provide the SHA1 Hash of the Desired Release of a Javascript Library. This would ensure a common identifier for the same source file across browsers that support the feature. This would be useful for developers as well. A debug tool can indicate to a developer that the script they are using is available in the Browser Repository with a certain identifier. This isn't a pro -- it's additional work for the standards body You are correct. I'm not too familiar with the process behind the specifications (although I would like to learn). Maybe including this as a "Pro" was premature, but having a hash value like a SHA1 be the identifier has a number of advantages. Maybe there are better solutions out there that have the same advantages that a SHA1 would provide. Cons: - May Not Grow Fast Enough: If JS Libraries change too quickly the repository won't get used enough. - May Not Scale: Are there too many JS Libraries, versions, etc making this unrealistic? Would storage become too large? - Adds significant spec complexity - Adds developer complexity, imagine a developer modifies their servers copy of a given script but forgets to update the references to the script, now they get inconsistent behaviour between browsers that support this feature and browsers that don't. Significant spec complexity? I'm too inexperienced to know. =( As for the developer scenario this is similar to modifying any single attribute on a tag and not appropriately modifying the others. Change the "src" on an and not changing the "alt". Change the "href" on a and not changing the "media". I think this could be easily avoided with validation. If the unique identifier were a SHA1 hash and the referenced script "src" does not hash to the provided value, then the page would invalidate with an error/warning. However, you raise a good point, and I can't come up with any truly equivalent analogy to any other existing developer specific problem. --Oliver Thanks for the points Oliver. Looks like a few of the Pros may have been eliminated. Do you think this could produce any noticeable improvements? - Joe
[whatwg] Browser Bundled Javascript Repository
Hey Guys, This is my first time on the list, I searched the Archives but I didn't see anything like this so I apologize if I missed any earlier discussion on something like this. A while back I came across this two paragraph blog post titled "Browsers Should Bundle JS Libraries:" http://fukamachi.org/wp/2009/03/30/browsers-should-bundle-js-libraries/ The premise is basically that browsers are repeatedly downloading the same javascript frameworks from different domains over and over every day. In the author's own words: "All popular, stable Javascript libraries, all open source. All downloaded tens of millions of times a day, identical code each time." Below is a summary and expansion of my comments/ideas from the discussion on the above blog article. A typical solution to the problem, and one that works right now in browsers, is that if you require a javascript library on your website you can point to a "publicly available" version of that library. If enough sites use this public URI then the browser will continually be using that URI and it will be cached and reused by the browser. This is the idea behind Google's Hosted Libraries: http://code.google.com/apis/ajaxlibs/ There are some arguments against using Google's Hosted Libraries: http://www.derekallard.com/blog/post/jquery-hosted-on-google-and-some-implications-for-developers/ However, I think the author makes a good point. Bundling the JS Libraries in the Browser seems like it would require very little space, could even be stored in a more efficient representation (compiled bytecode for example), and would prevent an extra HTTP Request. The problem then becomes how does a browser know example.com's jquery.js is the same as other.com's jquery.js. The developer should opt-in to telling the browser it wants to use a certain JS Library version that the browser may already know about. The way I thought about it was by adding an attribute to the