Re: Modules and dependencies known before the load
On Fri, Aug 15, 2014 at 2:43 PM, Ian Hickson wrote: > On Fri, 15 Aug 2014, John Barton wrote: > > On Fri, Aug 15, 2014 at 9:43 AM, Ian Hickson wrote: > > > On Thu, 14 Aug 2014, John Barton wrote: > > > > > > > > But since the only way the client can know that it needs a.js and > > > > jquery.js is if the server tells it [...] > > > > > > There's at least four ways this can happen: > > > > > > - the server tells the browser that it needs file a.js when the > server is > > >sending index.html, then tells the browser that it needs query.js > when > > >it is sending a.js, and the server doesn't preemptively send > anything. > > > > > > - the server sends the browser all three files at once, preemptively. > > > > > > - the server tells the browser that it needs file a.js when sending > > >index.html, then when a.js is requested, it preemptively sends > > >jquery.js at the same time as telling the browser that it'l need it. > > > > > > - the server tells the browser that it needs files a.js and jquery.js > at > > >the same time as when it is sending index.html, and the server > doesn't > > >preemptively send anything. > > > > > > The first is too slow, since it requires one RTT per file. It's what we > > > have now, and what ES6 modules describe in the absence of server > support. > > > > > > The second is too slow, because it implies sending resources that > aren't > > > needed, possibly using up bandwidth that could be used for other > purposes. > > > It's something that can be supported by packaging and other solutions > > > already being developed, if people want it. > > > > The second method is faster than your method. It results in the same > > three file transfer in one less round trip. > > > > The second method does not imply sending any more or less resources than > > any other method listed. > > The second is too slow because there's lots of other files involved in > practice, for example the default style sheet, the scripts that are needed > straight away, the Web components that are needed straight away, all the > default images, etc. The whole point of the discussion in this thread is > that we're talking about files that are not needed straight away, and how > to obtain them promptly once they are needed. > Loading all of the files in a web page in the order they are needed is great goal and one that I think would make a great improvement in the Web. It is certainly true that data for pages is "chunky": we need a mechanism for loading a group of related files at the right time. Within each chunk we need all of the files if we need the root of the chunk. But that is exactly the ES6 case: if we need the root of the dependency tree, then we need all of the tree, that is the declarative design. Having a design where the browser gets all the names, sends all the names back to the server, and gets the tree is just wasting a trip. Simply send the tree when the browser asks for the tree. That's why in my opinion 'bundles' or 'packages' make sense: they combine the related dependencies and allow them to be loaded in one trip. Divide this problem in to small pieces: ES6 bundles, HTML Imports, and some bundle/package loading solution. Don't use the same fine-grained solution for all layers. > > > > > The second and third require server-side support beyond what many > > > authors will be able to do. HTTP2 can provide this if people want it, > > > I believe. > > > > > > The fourth is what I'm looking at. > > > > > > The fourth consists of the server having no built-in knowledge except > > > what is in-band in the HTML file, namely, predeclared dependency > > > trees. > > > > By humans writing markup? That's not happening, at least not for more > > than trivial programs. > > It turns out that on the Web, there are lots of "trivial" programs. When > you have trillions of Web pages, even the smallest of fractions ends up > being significant numbers of pages. > Such programs don't need the kind of features we are discussing. > > > > No one is going to write hundreds of lines of dependency declaration > > already specified in their JS files. > > Maybe not hundreds, but maybe dozens. Also, not all these dependencies are > written in their JS files. Also, it turns out that people will do lots of > things to make their pages quicker, and it's a lot easier, in many cases, > to adjust their markup than to adjust their server configuration. > But the JS ones are written in JS files. And the non-JS ones are written somewhere. Inventing a mechanism that causes devs to duplicate all of this info is crazy. Just to be clear: I think dependency loading web pages is an awesome direction. Using a duplicate declarative solution? Not so much. > > > > And if you allow a tool running on the server (eg a build) then its the > > same as bundling with an extra unnecessary round trip. > > The tool is likely to not be running on the server, but on the developent > machine. It's unfortunate,
Re: Modules and dependencies known before the load
On Fri, 15 Aug 2014, John Barton wrote: > On Fri, Aug 15, 2014 at 9:43 AM, Ian Hickson wrote: > > On Thu, 14 Aug 2014, John Barton wrote: > > > > > > But since the only way the client can know that it needs a.js and > > > jquery.js is if the server tells it [...] > > > > There's at least four ways this can happen: > > > > - the server tells the browser that it needs file a.js when the server is > >sending index.html, then tells the browser that it needs query.js when > >it is sending a.js, and the server doesn't preemptively send anything. > > > > - the server sends the browser all three files at once, preemptively. > > > > - the server tells the browser that it needs file a.js when sending > >index.html, then when a.js is requested, it preemptively sends > >jquery.js at the same time as telling the browser that it'l need it. > > > > - the server tells the browser that it needs files a.js and jquery.js at > >the same time as when it is sending index.html, and the server doesn't > >preemptively send anything. > > > > The first is too slow, since it requires one RTT per file. It's what we > > have now, and what ES6 modules describe in the absence of server support. > > > > The second is too slow, because it implies sending resources that aren't > > needed, possibly using up bandwidth that could be used for other purposes. > > It's something that can be supported by packaging and other solutions > > already being developed, if people want it. > > The second method is faster than your method. It results in the same > three file transfer in one less round trip. > > The second method does not imply sending any more or less resources than > any other method listed. The second is too slow because there's lots of other files involved in practice, for example the default style sheet, the scripts that are needed straight away, the Web components that are needed straight away, all the default images, etc. The whole point of the discussion in this thread is that we're talking about files that are not needed straight away, and how to obtain them promptly once they are needed. > > The second and third require server-side support beyond what many > > authors will be able to do. HTTP2 can provide this if people want it, > > I believe. > > > > The fourth is what I'm looking at. > > > > The fourth consists of the server having no built-in knowledge except > > what is in-band in the HTML file, namely, predeclared dependency > > trees. > > By humans writing markup? That's not happening, at least not for more > than trivial programs. It turns out that on the Web, there are lots of "trivial" programs. When you have trillions of Web pages, even the smallest of fractions ends up being significant numbers of pages. > No one is going to write hundreds of lines of dependency declaration > already specified in their JS files. Maybe not hundreds, but maybe dozens. Also, not all these dependencies are written in their JS files. Also, it turns out that people will do lots of things to make their pages quicker, and it's a lot easier, in many cases, to adjust their markup than to adjust their server configuration. > And if you allow a tool running on the server (eg a build) then its the > same as bundling with an extra unnecessary round trip. The tool is likely to not be running on the server, but on the developent machine. It's unfortunate, and I don't really understand it, but it's a fact of life on the Web that many authors have only minimal control over their servers. > > > If you meant "suppose we had magic file on the server which listed > > > dependencies and a Loader that read that file, then we can succeed > > > by building the magic file and don't need server code changes". > > > > Well, the file wouldn't be magic. It'd just be a predeclared > > dependencies in the HTML file, something that authors have long asked > > for, at least on the WHATWG list. > > You mean devs re-writing the same declarations they used in JS in HTML? > Do these authors write modular JS? I am very skeptical. Currently, many of these developers don't write modular JS, no. However, my work here is intended to make it possible to use the ES6 module loading system even with legacy scripts, and to be able to declare dependencies for them too, so it's actually not a concern if it's not ES6 modules that they are writing. Indeed it might not be script at all. Declaring dependencies just between style sheets, HTML imports, images, and other resources will similarly require a dependency system, and if we reuse ES6's rather than require all browsers to implement two or more dependency systems, then the requirements for those non-script cases end up applying to ES6 modules. > > > That's what "bundling" does: builds a magic file and ships all of > > > the required modules in dependency order to the server. It even > > > skips the step where the list it built and just sends the all th
Re: Modules and dependencies known before the load
> > > I think it would be reasonable for us to say that all the dependency > declaration mechanisms are equivalent in that they all cause the target > dependency to be "executed" (whatever that means in the context). OK. The "instantiate" hook allows you to specify arbitrary dependencies, but only if the module in question is not coming from ES6 source code. There's currently no way to say: "instantiate this module from source as a normal ES6 module, but add A, B, and C to the dependency list." AFAIK, anyway. > This > might need careful management around some resource types, though. In > particular, CSS imports don't de-dupe, so we'd have to say what it means > if you do: > > > @import url(my-other-stylesheet); > > > Does this cause "my-other-stylesheet" to be loaded twice? > This is from your other post, right? If you're going to use the ES module loader as a general dependency management framework, then each resource will need to have a corresponding ES module "image". What if the corresponding ES module for a stylesheet, instead of initializing and exposing a stylesheet object, exposed a factory function for creating stylesheet objects? > But that's probably out of scope for this list. > Sometimes it seems like much of ES is out of scope for this list : ) ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
On Fri, Aug 15, 2014 at 9:43 AM, Ian Hickson wrote: > On Thu, 14 Aug 2014, John Barton wrote: > > > > But since the only way the client can know that it needs a.js and > > jquery.js is if the server tells it [...] > > There's at least four ways this can happen: > > - the server tells the browser that it needs file a.js when the server is >sending index.html, then tells the browser that it needs query.js when >it is sending a.js, and the server doesn't preemptively send anything. > > - the server sends the browser all three files at once, preemptively. > > - the server tells the browser that it needs file a.js when sending >index.html, then when a.js is requested, it preemptively sends >jquery.js at the same time as telling the browser that it'l need it. > > - the server tells the browser that it needs files a.js and jquery.js at >the same time as when it is sending index.html, and the server doesn't >preemptively send anything. > > The first is too slow, since it requires one RTT per file. It's what we > have now, and what ES6 modules describe in the absence of server support. > > The second is too slow, because it implies sending resources that aren't > needed, possibly using up bandwidth that could be used for other purposes. > It's something that can be supported by packaging and other solutions > already being developed, if people want it. > The second method is faster than your method. It results in the same three file transfer in one less round trip. The second method does not imply sending any more or less resources than any other method listed. > > The second and third require server-side support beyond what many authors > will be able to do. HTTP2 can provide this if people want it, I believe. > > The fourth is what I'm looking at. > > The fourth consists of the server having no built-in knowledge except > what is in-band in the HTML file, namely, predeclared dependency trees. > By humans writing markup? That's not happening, at least not for more than trivial programs. No one is going to write hundreds of lines of dependency declaration already specified in their JS files. And if you allow a tool running on the server (eg a build) then its the same as bundling with an extra unnecessary round trip. > ... > > > If you meant "suppose we had magic file on the server which listed > > dependencies and a Loader that read that file, then we can succeed by > > building the magic file and don't need server code changes". > > Well, the file wouldn't be magic. It'd just be a predeclared dependencies > in the HTML file, something that authors have long asked for, at least on > the WHATWG list. > You mean devs re-writing the same declarations they used in JS in HTML? Do these authors write modular JS? I am very skeptical. > > > > That's what "bundling" does: builds a magic file and ships all of the > > required modules in dependency order to the server. It even skips the > > step where the list it built and just sends the all the dependencies. > > > > Could we have better tools that allowed finer grained control over the > > bundle size vs trips? Sure. > > Ok, great. That's all I'm asking for. Actually I'm asking for less than > that, I'm asking if the ES6 module system could be slightly adjusted so > that this finer-grained control could be implemented on top of it without > duplicating the dependency management (which is what you have to do now > if you manually call LoadModule() on the dependencies early). > You can specify dependencies in `instantiate()` returns. jjb ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
On Fri, 15 Aug 2014, Kevin Smith wrote: > > > > Can you elaborate on this, just so I'm sure I understand it right? > > The execution order is determined by a depth-first traversal of the > dependency graph starting with the node being loaded, where edges are > ordered. So adding edges can have an effect on the traversal ordering. > Execution order is in general not fixed, though, since the user can > initiate loads of different parts of the graph at arbitrary times. > Just something to consider. Ok, cool, that's what I thought. Thanks. > > > 2) The ability to prefetch resources, given a URL which is currently > > > loading. > > As Juan pointed out (and I forgot) this can be done with > Loader.prototype.load. The problem with doing this is that it requires that the dependency tree be redundantly managed in two places: once on the HTML side, and once on the ES6 side. I would really like to avoid browsers having to implement two more or less identical dependency systems. > I think there's an important distinction here. Let's say that you have > this dependency chain: > > A -> B -> C > > So A depends on both B and C. > > Let's say that the user has declaratively indicated that A also depends > on some other resource D, but A doesn't *actually* depend on D. How > should things be modeled in that case? Would it make sense to add an > edge from A to D? > > If D is an "actual" dependency, then you would want to > execute/initialize it before A. On the other hand, if it's just a > "likely" dependency, then you might want to load it (and it's > dependencies) without initializing it, using `Loader.prototype.load`, or > perhaps some other mechanism. Ah, I see. I think it would be reasonable for us to say that all the dependency declaration mechanisms are equivalent in that they all cause the target dependency to be "executed" (whatever that means in the context). This might need careful management around some resource types, though. In particular, CSS imports don't de-dupe, so we'd have to say what it means if you do: @import url(my-other-stylesheet); Does this cause "my-other-stylesheet" to be loaded twice? But that's probably out of scope for this list. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
On Thu, 14 Aug 2014, John Barton wrote: > > But since the only way the client can know that it needs a.js and > jquery.js is if the server tells it [...] There's at least four ways this can happen: - the server tells the browser that it needs file a.js when the server is sending index.html, then tells the browser that it needs query.js when it is sending a.js, and the server doesn't preemptively send anything. - the server sends the browser all three files at once, preemptively. - the server tells the browser that it needs file a.js when sending index.html, then when a.js is requested, it preemptively sends jquery.js at the same time as telling the browser that it'l need it. - the server tells the browser that it needs files a.js and jquery.js at the same time as when it is sending index.html, and the server doesn't preemptively send anything. The first is too slow, since it requires one RTT per file. It's what we have now, and what ES6 modules describe in the absence of server support. The second is too slow, because it implies sending resources that aren't needed, possibly using up bandwidth that could be used for other purposes. It's something that can be supported by packaging and other solutions already being developed, if people want it. The second and third require server-side support beyond what many authors will be able to do. HTTP2 can provide this if people want it, I believe. The fourth is what I'm looking at. The fourth consists of the server having no built-in knowledge except what is in-band in the HTML file, namely, predeclared dependency trees. > > The key is just that the client can request all the files it's going > > to need at one time, so that, even without server support, the client > > doesn't wait more than one total RTT. > > Wait, you claimed earlier that without server support it's extra round > trips. Now without server support it's no extra round trips. Which one > ;-) As specced in ES6, it's one extra round trip per depth in the dependency chart; with predeclared dependencies, which is what I would like the ES6 module system to enable me to support in the HTML spec, it would be one RTT per branch of the dependency tree that is activated at once. > If you meant "suppose we had magic file on the server which listed > dependencies and a Loader that read that file, then we can succeed by > building the magic file and don't need server code changes". Well, the file wouldn't be magic. It'd just be a predeclared dependencies in the HTML file, something that authors have long asked for, at least on the WHATWG list. > That's what "bundling" does: builds a magic file and ships all of the > required modules in dependency order to the server. It even skips the > step where the list it built and just sends the all the dependencies. > > Could we have better tools that allowed finer grained control over the > bundle size vs trips? Sure. Ok, great. That's all I'm asking for. Actually I'm asking for less than that, I'm asking if the ES6 module system could be slightly adjusted so that this finer-grained control could be implemented on top of it without duplicating the dependency management (which is what you have to do now if you manually call LoadModule() on the dependencies early). -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
Let me clarify that when I suggest a mechanism for execution phase I am really asking for a loader hook that could allow user code to enable such a thing. Not asking that it be built into the loader spec. - Matthew Robb On Fri, Aug 15, 2014 at 9:56 AM, Matthew Robb wrote: > In YUI's Loader config you can set up hooks that specify another module > that "triggers" this module. It's the reverse of a dependency. You may want > to say that ModuleX should always be loaded before ModuleA-ModuleD. You > might even want to create this dependency based on environment conditions. > The common case here is polyfills but I have used it for other things as > well. > > I am curious what the reasoning is to not provide a mechanism in the > loader to allow the dependency graph to be adjusted in some manner? It > seems reasonable to me that you should be able to add dependencies to a > module at any time. These are not imported dependencies just load and > potentially execution dependencies and it WOULD be great if this mechanism > provided a model for specifying execution phase. Again similar to YUI's > "before", "after", and "instead" values for the "when" property of its > condition blocks in loader configuration. > > > - Matthew Robb > > > On Fri, Aug 15, 2014 at 9:27 AM, Kevin Smith wrote: > >> >>> > The structure of the graph has an effect on order-of-execution, though, >>> > so adding edges can result in observable changes. >>> >>> Can you elaborate on this, just so I'm sure I understand it right? >>> >>> >> The execution order is determined by a depth-first traversal of the >> dependency graph starting with the node being loaded, where edges are >> ordered. So adding edges can have an effect on the traversal ordering. >> Execution order is in general not fixed, though, since the user can >> initiate loads of different parts of the graph at arbitrary times. Just >> something to consider. >> >> >>> > 2) The ability to prefetch resources, given a URL which is currently >>> > loading. >>> >> >> As Juan pointed out (and I forgot) this can be done with >> Loader.prototype.load. >> >> >>> > >>> > For example, the user might want to specify up-front a portion of the >>> > dependency graph so that resources in that graph can be downloaded in >>> > parallel. >>> >>> Isn't that just something you can do one you have (1)? I'm not sure I'm >>> following the distinction. >>> >> >> I think there's an important distinction here. Let's say that you have >> this dependency chain: >> >> A -> B -> C >> >> So A depends on both B and C. >> >> Let's say that the user has declaratively indicated that A also depends >> on some other resource D, but A doesn't *actually* depend on D. How should >> things be modeled in that case? Would it make sense to add an edge from A >> to D? >> >> If D is an "actual" dependency, then you would want to execute/initialize >> it before A. On the other hand, if it's just a "likely" dependency, then >> you might want to load it (and it's dependencies) without initializing it, >> using `Loader.prototype.load`, or perhaps some other mechanism. >> >> >> ___ >> es-discuss mailing list >> es-discuss@mozilla.org >> https://mail.mozilla.org/listinfo/es-discuss >> >> > ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
In YUI's Loader config you can set up hooks that specify another module that "triggers" this module. It's the reverse of a dependency. You may want to say that ModuleX should always be loaded before ModuleA-ModuleD. You might even want to create this dependency based on environment conditions. The common case here is polyfills but I have used it for other things as well. I am curious what the reasoning is to not provide a mechanism in the loader to allow the dependency graph to be adjusted in some manner? It seems reasonable to me that you should be able to add dependencies to a module at any time. These are not imported dependencies just load and potentially execution dependencies and it WOULD be great if this mechanism provided a model for specifying execution phase. Again similar to YUI's "before", "after", and "instead" values for the "when" property of its condition blocks in loader configuration. - Matthew Robb On Fri, Aug 15, 2014 at 9:27 AM, Kevin Smith wrote: > >> > The structure of the graph has an effect on order-of-execution, though, >> > so adding edges can result in observable changes. >> >> Can you elaborate on this, just so I'm sure I understand it right? >> >> > The execution order is determined by a depth-first traversal of the > dependency graph starting with the node being loaded, where edges are > ordered. So adding edges can have an effect on the traversal ordering. > Execution order is in general not fixed, though, since the user can > initiate loads of different parts of the graph at arbitrary times. Just > something to consider. > > >> > 2) The ability to prefetch resources, given a URL which is currently >> > loading. >> > > As Juan pointed out (and I forgot) this can be done with > Loader.prototype.load. > > >> > >> > For example, the user might want to specify up-front a portion of the >> > dependency graph so that resources in that graph can be downloaded in >> > parallel. >> >> Isn't that just something you can do one you have (1)? I'm not sure I'm >> following the distinction. >> > > I think there's an important distinction here. Let's say that you have > this dependency chain: > > A -> B -> C > > So A depends on both B and C. > > Let's say that the user has declaratively indicated that A also depends on > some other resource D, but A doesn't *actually* depend on D. How should > things be modeled in that case? Would it make sense to add an edge from A > to D? > > If D is an "actual" dependency, then you would want to execute/initialize > it before A. On the other hand, if it's just a "likely" dependency, then > you might want to load it (and it's dependencies) without initializing it, > using `Loader.prototype.load`, or perhaps some other mechanism. > > > ___ > es-discuss mailing list > es-discuss@mozilla.org > https://mail.mozilla.org/listinfo/es-discuss > > ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
> > > > The structure of the graph has an effect on order-of-execution, though, > > so adding edges can result in observable changes. > > Can you elaborate on this, just so I'm sure I understand it right? > > The execution order is determined by a depth-first traversal of the dependency graph starting with the node being loaded, where edges are ordered. So adding edges can have an effect on the traversal ordering. Execution order is in general not fixed, though, since the user can initiate loads of different parts of the graph at arbitrary times. Just something to consider. > > 2) The ability to prefetch resources, given a URL which is currently > > loading. > As Juan pointed out (and I forgot) this can be done with Loader.prototype.load. > > > > For example, the user might want to specify up-front a portion of the > > dependency graph so that resources in that graph can be downloaded in > > parallel. > > Isn't that just something you can do one you have (1)? I'm not sure I'm > following the distinction. > I think there's an important distinction here. Let's say that you have this dependency chain: A -> B -> C So A depends on both B and C. Let's say that the user has declaratively indicated that A also depends on some other resource D, but A doesn't *actually* depend on D. How should things be modeled in that case? Would it make sense to add an edge from A to D? If D is an "actual" dependency, then you would want to execute/initialize it before A. On the other hand, if it's just a "likely" dependency, then you might want to load it (and it's dependencies) without initializing it, using `Loader.prototype.load`, or perhaps some other mechanism. ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
On Thu, Aug 14, 2014 at 2:20 PM, Ian Hickson wrote: > On Wed, 13 Aug 2014, John Barton wrote: > > On Wed, Aug 13, 2014 at 3:52 PM, Ian Hickson wrote: > > > On Wed, 13 Aug 2014, John Barton wrote: > > > > On Wed, Aug 13, 2014 at 2:59 PM, Ian Hickson wrote: > > > > > > > > > > Suppose a page has this markup: > > > > > > > > > > whenneeded> > > > > > whenneeded> > > > > > > > > > > import "a"; > > > > > // ... > > > > > > > > > > > > > Your example will just work if you just use > > > > > > > > > > > > import "a"; > > > > // ... > > > > > > > > > > > > When this module is compiled the other two will be loaded and > > > > compiled. > > > > > > That assumes that you have some sort of server-side support so that > > > the server knows that when you request "a", it should send "a.js" and > > > "jquery.js", which is an assumption that will work for some people but > > > not [...] > > > > No, the loader just requests each of these when it needs them. > > Right. What I'm trying to do is have the loader load them _before_ it > needs them. > > Without server-side support, on-demand loading looks like this: > > Client Server > || > 0ms |-_ | Client finds it needs a.js > | -_| > |-_ | > | -_| Server sends back a.js > | _- | > | _- | > | _- | > 200ms |-_ | Client finds it needs jquery.js > | -_| > |-_ | > | -_| Server sends back jquery.js > | _- | > | _- | > | _- | > 400ms |- | Client runs the scripts > > ...whereas if the client is aware of the dependencies early, it can do: > > Client Server > || > 0ms |-_ | Client finds it needs a.js and jquery.js > | -_| > |-_ | > | -_| Server sends back a.js and jquery.js > | _- | > | _- | > | _- | > 200ms |- | Client runs the scripts > > ...which is a lot faster. > But since the only way the client can know that it needs a.js and jquery.js is if the server tells it, then: Client Server sends a.js and jquery.js || 0ms |-_ | Client finds it has a.js and jquery.js 0ms |- | Client runs the scripts which is even faster. > > > > everyone. One of the big pieces of feedback I am dealing with on the > > > HTML side is a request for a way to predeclare the dependency graph to > > > avoid this problem (so that the browser knows to request a.js and > > > jquery.js at > > > > Actually I guess you mean "avoid multi-pass requests for the entire > > dependency graph". > > I'm not sure what that means. But if that means the same thing, then sure. > Yes, your example just happens to be one layer deep and thus requires just one extra trip. > The key is just that the client can request all the files it's going to > need at one time, so that, even without server support, the client doesn't > wait more than one total RTT. > Wait, you claimed earlier that without server support it's extra round trips. Now without server support it's no extra round trips. Which one ;-) If you meant "suppose we had magic file on the server which listed dependencies and a Loader that read that file, then we can succeed by building the magic file and don't need server code changes". That's what "bundling" does: builds a magic file and ships all of the required modules in dependency order to the server. It even skips the step where the list it built and just sends the all the dependencies. Could we have better tools that allowed finer grained control over the bundle size vs trips? Sure. > > > > > (If you don't have either server-side support or client-side > > > dependency predeclaration, you end up having to discover the > > > dependency chart at runtime, which is a huge latency loss.) > > > > Well, as they said back when I worked on a dairy, "you've step in it". > > > > This issue is at the root of the split between commonjs (modules for > > node, synchronous loading) and requirejs (modules for browser, prebuilt > > into bundles to avoid latency). The ES module system attempts to solve > > this by providing an asynchronous API (eg System.import()) and a > > proposal, to support server side dependency bundling in HTTP++. > > > > The workaround we have now is to pre-build bundles and use the locate() > > function to map module ids into bundles. That is basically the > > requirejs solution (with a much better programmer API). I predict node > > will extend their System to include a synchronous API. Then the circle > > will be complete. > > > > To be sure this is a difficult or even fundamental problem. I respect > > all efforts to solve this problem; don't believe anyone who says the > > solution is simple. > > Relying on packages doesn't really scale, unfortunately. When you have > thousands of modules, as e.g.
Re: Modules and dependencies known before the load
On Wed, 13 Aug 2014, John Barton wrote: > On Wed, Aug 13, 2014 at 3:52 PM, Ian Hickson wrote: > > On Wed, 13 Aug 2014, John Barton wrote: > > > On Wed, Aug 13, 2014 at 2:59 PM, Ian Hickson wrote: > > > > > > > > Suppose a page has this markup: > > > > > > > > > > > > > > > whenneeded> > > > > > > > > import "a"; > > > > // ... > > > > > > > > > > Your example will just work if you just use > > > > > > > > > import "a"; > > > // ... > > > > > > > > > When this module is compiled the other two will be loaded and > > > compiled. > > > > That assumes that you have some sort of server-side support so that > > the server knows that when you request "a", it should send "a.js" and > > "jquery.js", which is an assumption that will work for some people but > > not [...] > > No, the loader just requests each of these when it needs them. Right. What I'm trying to do is have the loader load them _before_ it needs them. Without server-side support, on-demand loading looks like this: Client Server || 0ms |-_ | Client finds it needs a.js | -_| |-_ | | -_| Server sends back a.js | _- | | _- | | _- | 200ms |-_ | Client finds it needs jquery.js | -_| |-_ | | -_| Server sends back jquery.js | _- | | _- | | _- | 400ms |- | Client runs the scripts ...whereas if the client is aware of the dependencies early, it can do: Client Server || 0ms |-_ | Client finds it needs a.js and jquery.js | -_| |-_ | | -_| Server sends back a.js and jquery.js | _- | | _- | | _- | 200ms |- | Client runs the scripts ...which is a lot faster. > > everyone. One of the big pieces of feedback I am dealing with on the > > HTML side is a request for a way to predeclare the dependency graph to > > avoid this problem (so that the browser knows to request a.js and > > jquery.js at > > Actually I guess you mean "avoid multi-pass requests for the entire > dependency graph". I'm not sure what that means. But if that means the same thing, then sure. The key is just that the client can request all the files it's going to need at one time, so that, even without server support, the client doesn't wait more than one total RTT. > > (If you don't have either server-side support or client-side > > dependency predeclaration, you end up having to discover the > > dependency chart at runtime, which is a huge latency loss.) > > Well, as they said back when I worked on a dairy, "you've step in it". > > This issue is at the root of the split between commonjs (modules for > node, synchronous loading) and requirejs (modules for browser, prebuilt > into bundles to avoid latency). The ES module system attempts to solve > this by providing an asynchronous API (eg System.import()) and a > proposal, to support server side dependency bundling in HTTP++. > > The workaround we have now is to pre-build bundles and use the locate() > function to map module ids into bundles. That is basically the > requirejs solution (with a much better programmer API). I predict node > will extend their System to include a synchronous API. Then the circle > will be complete. > > To be sure this is a difficult or even fundamental problem. I respect > all efforts to solve this problem; don't believe anyone who says the > solution is simple. Relying on packages doesn't really scale, unfortunately. When you have thousands of modules, as e.g. Google+ does, some of which are script and some are style sheets and some are sprite sheets and some are Web components and so on, you really need to be able to tell the server "we need these files", and you need to be able to do this without having to discover the dependencies once you've fetched the first batch of files. On Thu, 14 Aug 2014, Juan Ignacio Dopazo wrote: > > For prefetching we're calling LoadModule() for all the dependencies we > want to prefetch in the Fetch or Locate hooks. For example, Systemjs has > a System.depCache option which allows you to define dependencies ahead > of time. Here's the hook > implementation: > https://github.com/systemjs/systemjs/blob/master/lib/extension-depCache.js. Interesting, ok. So basically you have to manage the dependency tree outside of the ES6 Loader mechanism proper? I'm really hoping we can find a way to avoid the redundancy of having two mechanisms managing dependencies. On Wed, 13 Aug 2014, Kevin Smith wrote: > > I see two related abilities here, and I'm not quite sure if you want one > of them or both: > > 1) The ability to add nodes (and edges) to the dependency graph as the > graph is traversed by load operations. > > For example, the user might declaratively indicate that a module depends > upon
Re: Modules and dependencies known before the load
> > For prefetching we're calling LoadModule() for all the dependencies we > want to prefetch in the Fetch or Locate hooks. > Ah - just saw this - cool. ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
> > > If I understand what you're saying correctly, then yes (assuming that > those dependencies are also acted upon promptly, rather than only after > "fetch" has returned). > I see two related abilities here, and I'm not quite sure if you want one of them or both: 1) The ability to add nodes (and edges) to the dependency graph as the graph is traversed by load operations. For example, the user might declaratively indicate that a module depends upon some stylesheet. You might want to add a node to the graph representing that stylesheet. The structure of the graph has an effect on order-of-execution, though, so adding edges can result in observable changes. 2) The ability to prefetch resources, given a URL which is currently loading. For example, the user might want to specify up-front a portion of the dependency graph so that resources in that graph can be downloaded in parallel. I don't think the current Loader spec allows either of those abilities. And it might be that the second doesn't really belong in the loader proper. ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
For prefetching we're calling LoadModule() for all the dependencies we want to prefetch in the Fetch or Locate hooks. For example, Systemjs has a System.depCache option which allows you to define dependencies ahead of time. Here's the hook implementation: https://github.com/systemjs/systemjs/blob/master/lib/extension-depCache.js. - Juan___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
On Wed, Aug 13, 2014 at 3:52 PM, Ian Hickson wrote: > On Wed, 13 Aug 2014, John Barton wrote: > > On Wed, Aug 13, 2014 at 2:59 PM, Ian Hickson wrote: > > > > > > Suppose a page has this markup: > > > > > > > > > whenneeded> > > > > > > import "a"; > > > // ... > > > > > > > Your example will just work if you just use > > > > > > import "a"; > > // ... > > > > > > When this module is compiled the other two will be loaded and compiled. > > That assumes that you have some sort of server-side support so that the > server knows that when you request "a", it should send "a.js" and > "jquery.js", which is an assumption that will work for some people but not > No, the loader just requests each of these when it needs them. > everyone. One of the big pieces of feedback I am dealing with on the HTML > side is a request for a way to predeclare the dependency graph to avoid > this problem (so that the browser knows to request a.js and jquery.js at > Actually I guess you mean "avoid multi-pass requests for the entire dependency graph". > the same time). People apparently have huge difficulties even getting > their servers configured to send the right MIME types, let alone getting > the servers to know about dependencies. > Yup. > > (If you don't have either server-side support or client-side dependency > predeclaration, you end up having to discover the dependency chart at > runtime, which is a huge latency loss.) > Well, as they said back when I worked on a dairy, "you've step in it". This issue is at the root of the split between commonjs (modules for node, synchronous loading) and requirejs (modules for browser, prebuilt into bundles to avoid latency). The ES module system attempts to solve this by providing an asynchronous API (eg System.import()) and a proposal, to support server side dependency bundling in HTTP++. The workaround we have now is to pre-build bundles and use the locate() function to map module ids into bundles. That is basically the requirejs solution (with a much better programmer API). I predict node will extend their System to include a synchronous API. Then the circle will be complete. To be sure this is a difficult or even fundamental problem. I respect all efforts to solve this problem; don't believe anyone who says the solution is simple. jjb ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
On Wed, 13 Aug 2014, Kevin Smith wrote: > > > > Ideally I think we should adjust the ES6 module system to support > > loading and compiling code (though not necessarily executing it) for > > dependencies at or around the "fetch" hook. > > It seems like it should be possible, in principle, to add dependencies > to a load record at any time prior the instantiate hook resolving. > After all, the effect of the instantiate hook is to make the dependency > list completely arbitrary. > > Would that solve the issue from your point of view? If I understand what you're saying correctly, then yes (assuming that those dependencies are also acted upon promptly, rather than only after "fetch" has returned). -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
> > > Ideally I think we should adjust the ES6 module system to support loading > and compiling code (though not necessarily executing it) for dependencies > at or around the "fetch" hook. > It seems like it should be possible, in principle, to add dependencies to a load record at any time prior the instantiate hook resolving. After all, the effect of the instantiate hook is to make the dependency list completely arbitrary. Would that solve the issue from your point of view? ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
On Wed, 13 Aug 2014, John Barton wrote: > On Wed, Aug 13, 2014 at 2:59 PM, Ian Hickson wrote: > > > > Suppose a page has this markup: > > > > > > > > > > import "a"; > > // ... > > > > Your example will just work if you just use > > > import "a"; > // ... > > > When this module is compiled the other two will be loaded and compiled. That assumes that you have some sort of server-side support so that the server knows that when you request "a", it should send "a.js" and "jquery.js", which is an assumption that will work for some people but not everyone. One of the big pieces of feedback I am dealing with on the HTML side is a request for a way to predeclare the dependency graph to avoid this problem (so that the browser knows to request a.js and jquery.js at the same time). People apparently have huge difficulties even getting their servers configured to send the right MIME types, let alone getting the servers to know about dependencies. (If you don't have either server-side support or client-side dependency predeclaration, you end up having to discover the dependency chart at runtime, which is a huge latency loss.) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Modules and dependencies known before the load
Your example will just work if you just use import "a"; // ... When this module is compiled the other two will be loaded and compiled. jjb On Wed, Aug 13, 2014 at 2:59 PM, Ian Hickson wrote: > > One of the problems I'm running into when it comes to trying to integrate > ES6 modules with HTML and new HTML-based dependency features is the way > that I can't tell ES about dependencies I know about before the data is > actually fetched and instantiated. The problem can essentially be > summarised by this line from the definition of ProcessLoadDependencies(): > > # The ProcessLoadDependencies abstract operation is called after one > # module has nearly finished loading. It starts new loads as needed to > # load the module's dependencies. > > Suppose a page has this markup: > > > > > import "a"; > // ... > > > ...where "uses" is some hypothetical markup for telling the browser about > dependencies ahead of time (the assumption being that anims.js contains > a line like 'import "jquery";'), and where "whenneeded" is some attribute > that tells the browser to not bother starting the whole loading process > until the resource is needed. > > In this example, the first two
