Re: Shepherd.js - Implementing Harmony modules for today's browsers
@David: I am all ears to your comments! @Erik: As you may have read in this thread, it seems that the major request is for the removal of comments. So, here I announce it: the feature has just bumped to the top of the TODO list :) Thanks for the pointers. I'll probably have a closer look at the functional coverage of Traceur, as it has clearly evolved since I last watched it. On Wed, Apr 18, 2012 at 11:06 PM, Erik Arvidsson erik.arvids...@gmail.comwrote: This is quite interesting. I really feel that the comment syntax is pretty ugly. Parsing JS is non trivial but it is not a performance issue. If you expect this to have some uptake I would expect it to use the real module syntax and not rely on comments. By coincidence I landed import support to Traceur last night. You might want want to check it out. It also doesn't use the latest syntax since the current BNF on the wiki is incomplete. http://code.google.com/p/traceur-compiler/source/browse/#git%2Ftest%2Ffeature%2FModules http://code.google.com/p/traceur-compiler/source/detail?r=f4f8788860f624ca1b02883890325cbb4ee9c1eb Traceur does have a CodeLoader that allows loading external js files but it is not very convenient to use at this point. We have an open bug to allow offline compilation of modules with external dependancies. http://code.google.com/p/traceur-compiler/issues/detail?id=87 -- erik -- -- Xavier CAMBAR @xcambar https://twitter.com/#!/xcambar T: +33 6.84.29.46.83 ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Shepherd.js - Implementing Harmony modules for today's browsers
On Wed, Apr 18, 2012 at 10:21 AM, Xavier CAMBAR xcam...@gmail.com wrote:
Thanks for your comments, I'm glad you liked it.
About CommonJS, the compatibility is the other way round. Shepherd can
load commonJS modules without the addition of the in-comment syntax
declaration. In such a case, require is wrapped to load whether an already
loaded ES6 module or use commonJS's require. furthermore, exports or
module.exports is used as the public API of the ES6 module (which is
basically the definition :) ). This is really useful for 3rd party modules.
Removing the comments would be possible, but it would require a much
heavier parsing phase, and would probably require a full-blown JS parser,
such as Esprima. it would be probably computationnally too heavy for the
browser (read: for the user) without a systematic and automatic
code-rewrite to ES5, optionnaly accompagned with some minification.
Yes, that's what I was getting at in my response. I know it would be a much
bigger effort, which is why I understand the approach. If I were to do the
project, though, I would assume that if it were done in the browser, that
would be purely for development - it should all be precompiled for
production. You could even use the module imports to do script
concatenation.
On the other hand, the great advantage of using comments is that they act
as placeholders for the syntax declaration, and are very easy to locate
with a single regular expression.
This is beneficial for the implementation, but not the user.
More realistically, I plan on allowing multiple module declarations in a
single file (currently one only), where module implementations would be
syntactically separated by the numerous definitions (ie, the implementation
of a module ends at the beginning of the next module declaration).
But yes, I am looking for an efficient way to remove the comments, which
would be the only way Shepherd could be used as an efficient
Harmony:modules polyfill. Maybe I'll have to play around with more
minimalist placeholders, such as:
Personally, I think you'll want the full polyfill as a target. You can get
there incrementally by removing comments, but until it matches the full
spec, I don't think you'll see widespread adoption. IMHO anyway.
//s6d
module myModule {
import x from X;
export a, b, c, d;
}
//-s6d
...though the ending comment doesn't seem necessary.
And please don't blame me for the possible terrible idea, I'm thinking
aloud on this one! ;)
I think I would actually prefer comments over this. I wrote some code long
before module proposals that basically made my JavaScript files work like
Java files (Java is the other language we use), basically import
statements, exports were just based on the filename matching the class or
object definition inside. I would probably change it to be more like
modules if I did it now, maybe even how you did it. Anyway, point is, I put
the imports in the comments just like you for two reasons.
1. easier to find with regexes, didn't have to parse full js
2. didn't break existing JavaScript tooling i.e. syntax highlighting
Its the second reason that I would prefer your current comment based
solution over the new one you propose. Unless you support writing modules
the way I expect them to be written, I would rather have them in comments.
The tradeoff to be found is the following: As of today, no browser has a
stable release that allows harmony modules (yeah, the latest V8 has an
option, well...). Whatsmore, the syntax is just a proposal which is updated
almost every quarter. So do we want to allow files that work on today's
engines and can be enhanced with Harmony:modules' features thankd to
Shepherd, or do we want Harmony compliant files that won't run natively on
any stable engine we can find today?
I chose the first path, but the discussion remains open. I'll be happy to
hear your thoughts on that point.
Support is coming. I look at it this way. Some day relatively soon, ES6
modules will be in node. Soon after that they will start showing up in
browsers. Over the course of the next year, I bet you'll see module support
in at least firefox and chrome. If you set your sights on full modules now,
you'll have code that doesn't need to be rewritten a year from now. A
polyfill will happen, do you want to write it?
- Russ
Xavier
On Wed, Apr 18, 2012 at 3:11 PM, Russell Leggett
russell.legg...@gmail.com wrote:
This is great! I've been considering doing the same thing, but I haven't
found the time. When you say it is compatible with CommonJS modules, does
that mean that you can do an import using ES6 syntax and have the result do
a CommonJS require?
The big thing I'm noticing is that all of the examples are inside of
comments, and your module definitions including exports are all separate
from the actual code for those exports. I'm assuming that means somebody
couldn't actually take the ES6 module examples and
Re: Shepherd.js - Implementing Harmony modules for today's browsers
You've made solid points here!
The various comments I've received from here and there show the same
direction: someday, modules will be available, and Shepherd will then be
unnecessary for supported engines. Polyfilling remains. If it ever gains
some momentum, it is highly related to how it follows the specifications
and remains usable in supported engines.
It appears then that removing comments is a must-have. Keeping
backward-compatibility maybe was a bad idea from the very beginning after
all. If you look at the example of the home page (
xcambar.github.com/shepherd-js), who will seriously consider developing a
module with the relevant code architecture in mind, and then proclaim Hey,
you can also use it as a good old module-less JS file, but it will put crap
all over you global scope by exposing its inner parts ? I hope no one will.
As a conclusion, Shepherd seems to be going its way to a compliant polyfill
for Harmony modules. I should change the script type to text=harmony to
reflect this :p
But you shouldn't expect the tools you use not to break. Unless they parse
the modules syntax, of course! This is why I hardly understand your point
on the *proposed* new syntax. It's much more lightweight, is parsed easily
(*I* am the only one happy, but that matters at some point ;) ) and works
natively on module-enabled engines.
It may IMO constitute a valid first step, as it reduces the effort for the
user drastically.
Xavier
On Thu, Apr 19, 2012 at 3:11 PM, Russell Leggett
russell.legg...@gmail.comwrote:
On Wed, Apr 18, 2012 at 10:21 AM, Xavier CAMBAR xcam...@gmail.com wrote:
Thanks for your comments, I'm glad you liked it.
About CommonJS, the compatibility is the other way round. Shepherd can
load commonJS modules without the addition of the in-comment syntax
declaration. In such a case, require is wrapped to load whether an already
loaded ES6 module or use commonJS's require. furthermore, exports or
module.exports is used as the public API of the ES6 module (which is
basically the definition :) ). This is really useful for 3rd party modules.
Removing the comments would be possible, but it would require a much
heavier parsing phase, and would probably require a full-blown JS parser,
such as Esprima. it would be probably computationnally too heavy for the
browser (read: for the user) without a systematic and automatic
code-rewrite to ES5, optionnaly accompagned with some minification.
Yes, that's what I was getting at in my response. I know it would be a
much bigger effort, which is why I understand the approach. If I were to do
the project, though, I would assume that if it were done in the browser,
that would be purely for development - it should all be precompiled for
production. You could even use the module imports to do script
concatenation.
On the other hand, the great advantage of using comments is that they act
as placeholders for the syntax declaration, and are very easy to locate
with a single regular expression.
This is beneficial for the implementation, but not the user.
More realistically, I plan on allowing multiple module declarations in a
single file (currently one only), where module implementations would be
syntactically separated by the numerous definitions (ie, the implementation
of a module ends at the beginning of the next module declaration).
But yes, I am looking for an efficient way to remove the comments, which
would be the only way Shepherd could be used as an efficient
Harmony:modules polyfill. Maybe I'll have to play around with more
minimalist placeholders, such as:
Personally, I think you'll want the full polyfill as a target. You can get
there incrementally by removing comments, but until it matches the full
spec, I don't think you'll see widespread adoption. IMHO anyway.
//s6d
module myModule {
import x from X;
export a, b, c, d;
}
//-s6d
...though the ending comment doesn't seem necessary.
And please don't blame me for the possible terrible idea, I'm thinking
aloud on this one! ;)
I think I would actually prefer comments over this. I wrote some code long
before module proposals that basically made my JavaScript files work like
Java files (Java is the other language we use), basically import
statements, exports were just based on the filename matching the class or
object definition inside. I would probably change it to be more like
modules if I did it now, maybe even how you did it. Anyway, point is, I put
the imports in the comments just like you for two reasons.
1. easier to find with regexes, didn't have to parse full js
2. didn't break existing JavaScript tooling i.e. syntax highlighting
Its the second reason that I would prefer your current comment based
solution over the new one you propose. Unless you support writing modules
the way I expect them to be written, I would rather have them in comments.
The tradeoff to be found is the following: As of today, no browser
Re: Shepherd.js - Implementing Harmony modules for today's browsers
On Thu, Apr 19, 2012 at 9:11 AM, Russell Leggett
russell.legg...@gmail.comwrote:
[snip]
Support is coming. I look at it this way. Some day relatively soon, ES6
modules will be in node. Soon after that they will start showing up in
browsers. Over the course of the next year, I bet you'll see module support
in at least firefox and chrome.
In Chrome Canary, with Harmony flag enabled...
module Foo {}
(Foo in this); // true
:)
But, Foo is still undefined.
Rick
If you set your sights on full modules now, you'll have code that doesn't
need to be rewritten a year from now. A polyfill will happen, do you want
to write it?
- Russ
[snip]
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: Shepherd.js - Implementing Harmony modules for today's browsers
Russell,
I've worked on this branch during the evening:
https://github.com/xcambar/shepherd-js/tree/20120420_CommentRemoval,
Currently, single line comments must still wrap the module declaration, but
this version offers a good comparison point in regards to the master branch.
Rick,
I've seen this, it is encouraging that Google considers adding support. But
there's a long way to go before a stable release.
By the way, the V8 engine what will come in Node 0.8 has Harmony flags, and
one of them is about modules.
Xavier
On Thu, Apr 19, 2012 at 9:46 PM, Rick Waldron waldron.r...@gmail.comwrote:
On Thu, Apr 19, 2012 at 9:11 AM, Russell Leggett
russell.legg...@gmail.com wrote:
[snip]
Support is coming. I look at it this way. Some day relatively soon, ES6
modules will be in node. Soon after that they will start showing up in
browsers. Over the course of the next year, I bet you'll see module support
in at least firefox and chrome.
In Chrome Canary, with Harmony flag enabled...
module Foo {}
(Foo in this); // true
:)
But, Foo is still undefined.
Rick
If you set your sights on full modules now, you'll have code that
doesn't need to be rewritten a year from now. A polyfill will happen, do
you want to write it?
- Russ
[snip]
--
--
Xavier CAMBAR
@xcambar https://twitter.com/#!/xcambar
T: +33 6.84.29.46.83
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: Shepherd.js - Implementing Harmony modules for today's browsers
This is great! I've been considering doing the same thing, but I haven't found the time. When you say it is compatible with CommonJS modules, does that mean that you can do an import using ES6 syntax and have the result do a CommonJS require? The big thing I'm noticing is that all of the examples are inside of comments, and your module definitions including exports are all separate from the actual code for those exports. I'm assuming that means somebody couldn't actually take the ES6 module examples and make them work, for example. I appreciate the difference in difficulty level. Parsing just the module syntax vs being an ES5 compliant parser + modules is a much different task. The problem is, I don't see myself using it unless it actually used full module syntax. Do you plan on going in that direction? - Russ On Wed, Apr 18, 2012 at 8:20 AM, Xavier CAMBAR xcam...@gmail.com wrote: Hi, I wanted to announce that I've been working on a project called Shepherd ( http://xcambar.github.com/shepherd-js), a pure Javascript implementation of Harmony modules. Why such a project ? Fun first. Second, I was really looking forward to use harmony modules. Third, I wanted an efficient way to use my modules on the server and the client. And it seems to me that current module loaders and APIs available will be, at the end, superseded by the module syntax being defined at ECMA for the future versions of ECMAScript. The syntax used is as of 2012-02-27 ( http://wiki.ecmascript.org/doku.php?id=harmony:modulesrev=1330363672), I couldn't find the time to implement the latest proposal (besides I would have had to choose one of the two variants), but apart from the syntax, it is usable both on the client and the server, and tested. The parser/lexer has been developped using JISON ( http://zaach.github.com/jison/) and is available as a separate project ( https://github.com/xcambar/harmony-parser). For backward-compatibility, the module declarations have to be put into comments (which you will discover in the examples provided on the site), it is compatible with CommonJS modules (user-defined as well as native modules in Node.js), and, although not critical to the project, a compatibility wrapper for the AMD API is on its way. Regarding production-level requirements, an optimizer has been implemented, but it still requires testing before being released. I've had and I'm still having a really good time working on this project and I would really appreciate if I could have some feedback from the readers and contributors of the mailing-list. A mailing list has been created for the project, still empty of messages simply because it has been created yesterday ;) Regards, -- Xavier CAMBAR @xcambar https://twitter.com/#!/xcambar ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Shepherd.js - Implementing Harmony modules for today's browsers
Thanks for your comments, I'm glad you liked it.
About CommonJS, the compatibility is the other way round. Shepherd can load
commonJS modules without the addition of the in-comment syntax declaration.
In such a case, require is wrapped to load whether an already loaded ES6
module or use commonJS's require. furthermore, exports or module.exports is
used as the public API of the ES6 module (which is basically the definition
:) ). This is really useful for 3rd party modules.
Removing the comments would be possible, but it would require a much
heavier parsing phase, and would probably require a full-blown JS parser,
such as Esprima. it would be probably computationnally too heavy for the
browser (read: for the user) without a systematic and automatic
code-rewrite to ES5, optionnaly accompagned with some minification.
On the other hand, the great advantage of using comments is that they act
as placeholders for the syntax declaration, and are very easy to locate
with a single regular expression.
More realistically, I plan on allowing multiple module declarations in a
single file (currently one only), where module implementations would be
syntactically separated by the numerous definitions (ie, the implementation
of a module ends at the beginning of the next module declaration).
But yes, I am looking for an efficient way to remove the comments, which
would be the only way Shepherd could be used as an efficient
Harmony:modules polyfill. Maybe I'll have to play around with more
minimalist placeholders, such as:
//s6d
module myModule {
import x from X;
export a, b, c, d;
}
//-s6d
...though the ending comment doesn't seem necessary.
And please don't blame me for the possible terrible idea, I'm thinking
aloud on this one! ;)
The tradeoff to be found is the following: As of today, no browser has a
stable release that allows harmony modules (yeah, the latest V8 has an
option, well...). Whatsmore, the syntax is just a proposal which is updated
almost every quarter. So do we want to allow files that work on today's
engines and can be enhanced with Harmony:modules' features thankd to
Shepherd, or do we want Harmony compliant files that won't run natively on
any stable engine we can find today?
I chose the first path, but the discussion remains open. I'll be happy to
hear your thoughts on that point.
Xavier
On Wed, Apr 18, 2012 at 3:11 PM, Russell Leggett
russell.legg...@gmail.comwrote:
This is great! I've been considering doing the same thing, but I haven't
found the time. When you say it is compatible with CommonJS modules, does
that mean that you can do an import using ES6 syntax and have the result do
a CommonJS require?
The big thing I'm noticing is that all of the examples are inside of
comments, and your module definitions including exports are all separate
from the actual code for those exports. I'm assuming that means somebody
couldn't actually take the ES6 module examples and make them work, for
example. I appreciate the difference in difficulty level. Parsing just the
module syntax vs being an ES5 compliant parser + modules is a much
different task. The problem is, I don't see myself using it unless it
actually used full module syntax. Do you plan on going in that direction?
- Russ
On Wed, Apr 18, 2012 at 8:20 AM, Xavier CAMBAR xcam...@gmail.com wrote:
Hi,
I wanted to announce that I've been working on a project called Shepherd (
http://xcambar.github.com/shepherd-js), a pure Javascript implementation
of Harmony modules.
Why such a project ? Fun first. Second, I was really looking forward to
use harmony modules. Third, I wanted an efficient way to use my modules on
the server and the client. And it seems to me that current module loaders
and APIs available will be, at the end, superseded by the module syntax
being defined at ECMA for the future versions of ECMAScript.
The syntax used is as of 2012-02-27 (
http://wiki.ecmascript.org/doku.php?id=harmony:modulesrev=1330363672),
I couldn't find the time to implement the latest proposal (besides I would
have had to choose one of the two variants), but apart from the syntax, it
is usable both on the client and the server, and tested.
The parser/lexer has been developped using JISON (
http://zaach.github.com/jison/) and is available as a separate project (
https://github.com/xcambar/harmony-parser).
For backward-compatibility, the module declarations have to be put into
comments (which you will discover in the examples provided on the site), it
is compatible with CommonJS modules (user-defined as well as native modules
in Node.js), and, although not critical to the project, a compatibility
wrapper for the AMD API is on its way.
Regarding production-level requirements, an optimizer has been
implemented, but it still requires testing before being released.
I've had and I'm still having a really good time working on this project
and I would really appreciate if I could have some feedback from the
readers and
Re: Shepherd.js - Implementing Harmony modules for today's browsers
Hi Xavier, It's great to see this project! I'll take a closer look soon. I will see if I can help contribute to this. Dave On Apr 18, 2012, at 5:20 AM, Xavier CAMBAR wrote: Hi, I wanted to announce that I've been working on a project called Shepherd (http://xcambar.github.com/shepherd-js), a pure Javascript implementation of Harmony modules. Why such a project ? Fun first. Second, I was really looking forward to use harmony modules. Third, I wanted an efficient way to use my modules on the server and the client. And it seems to me that current module loaders and APIs available will be, at the end, superseded by the module syntax being defined at ECMA for the future versions of ECMAScript. The syntax used is as of 2012-02-27 (http://wiki.ecmascript.org/doku.php?id=harmony:modulesrev=1330363672), I couldn't find the time to implement the latest proposal (besides I would have had to choose one of the two variants), but apart from the syntax, it is usable both on the client and the server, and tested. The parser/lexer has been developped using JISON (http://zaach.github.com/jison/) and is available as a separate project (https://github.com/xcambar/harmony-parser). For backward-compatibility, the module declarations have to be put into comments (which you will discover in the examples provided on the site), it is compatible with CommonJS modules (user-defined as well as native modules in Node.js), and, although not critical to the project, a compatibility wrapper for the AMD API is on its way. Regarding production-level requirements, an optimizer has been implemented, but it still requires testing before being released. I've had and I'm still having a really good time working on this project and I would really appreciate if I could have some feedback from the readers and contributors of the mailing-list. A mailing list has been created for the project, still empty of messages simply because it has been created yesterday ;) Regards, -- Xavier CAMBAR @xcambar ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Shepherd.js - Implementing Harmony modules for today's browsers
This is quite interesting. I really feel that the comment syntax is pretty ugly. Parsing JS is non trivial but it is not a performance issue. If you expect this to have some uptake I would expect it to use the real module syntax and not rely on comments. By coincidence I landed import support to Traceur last night. You might want want to check it out. It also doesn't use the latest syntax since the current BNF on the wiki is incomplete. http://code.google.com/p/traceur-compiler/source/browse/#git%2Ftest%2Ffeature%2FModules http://code.google.com/p/traceur-compiler/source/detail?r=f4f8788860f624ca1b02883890325cbb4ee9c1eb Traceur does have a CodeLoader that allows loading external js files but it is not very convenient to use at this point. We have an open bug to allow offline compilation of modules with external dependancies. http://code.google.com/p/traceur-compiler/issues/detail?id=87 -- erik ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
