Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Thomas Heller]

> Yes.  Currently optimization is something we are doing when compiling 
> clojurescript, but perhaps *ideally* optimization could be done after 
> bundling when all the javascript is available.   However, I realize this 
> may not actually work very well in practice since most modern js 
> applications include dozens or hundreds of npm libraries that are not 
> closure ready and would break in unexpected ways.  Perhaps this is why 
> UglyifyJS is more common for webpack users?  UglyifyJS does dead code 
> elimination as well I believe, is it as good? probably not? is it good 
> enough? not sure.
>


UglifyJS is not good enough in my opinion, not even close. The JS world has 
"accepted" that you need to make lots of many small files to make it 
acceptable. We don't since we have Closure.

 

> Do we still need analyzer data after its compiled? Can't we just use the 
> javascript as a lib? I assume that google closure isn't written in cljs but 
> cljs has no problem using or optimizing it... perhaps the packages are 
> distributed as unoptimized google closure compatible js and the webpack 
> plugin turns google closure modules into commonjs modules? not sure. just 
> ideas. 
>


You need to analyzer data if you want to compile any .cljs. It is not 
needed at runtime (which is where the closure library comes in).

 

> Best optimized perhaps, but possibly not the best js dev ux.  Many js 
> developers don't want to run java as part of their build process and might 
> prefer to use uglifyjs even if the results are not as good.  
>

But many js devs do optimize and the best results would probably be to put 
> the unoptimized closure together with the unoptimized javascript and then 
> optimize everything together. But my guess is that wont work in practice. 
>  UglifyJS I think is safer than closure, as good? probably not, good 
> enough? not sure.  Your solution probably produces best optimization, but 
> its worth asking if thats what devs care about the most?  Slower build 
> process with more complexity vs a faster website.  There will be devs on 
> both sides I think. 
>


This is one place where it is far more important to optimize for production 
users not the developer UX. I care very little about the fact that my 
production build may take a while longer if the resulting .js my users have 
to download is substantially smaller with much better runtime performance. 
The fear of Java is unreasonable and mostly FUD. If I run builds with 
create-react-app or create-react-native-app they are frequently much slower 
than a :advanced compiled CLJS build.

There are still lots of places where we can optimize the dev UX but 
production concerns will always come first.

JS integration is still a work in progress and will get a whole lot better 
soon. Stay tuned. ;)

/thomas
 

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Thomas Heller]

Hey,

I understand what you want to achieve and I definitely want to make the 
whole thing easier but some issues don't have an easy solution.

TL;DR: It's complicated.

First we need to split compilation and bundling.

CLJS->JS compilation just takes one .cljs file and compiles it to .js. To 
do that it needs analyzer data of all CLJS dependencies. If you were to 
publish compiled CLJS to npm you'd need to also ship the analyzer data 
somehow. The other problem is that certain compiler options affect the 
generated JS which means that for every .cljs file there are many possible 
.js results. Another thing is that you also need to account for the 
compiler version since that may also affect the output or the analyzer data 
which then may affect other files.

The second part is bundling things together, which means reducing the 
number of individual files down to a more reasonable number. shadow-cljs 
enables JS tools to do that for us (eg. webpack, create-react-app or any 
other), since those tools use "require" to discover dependencies.

The best option for this however is to use the Closure Compiler but to get 
the whole benefit of it you need to optimize your whole program, which 
includes EVERY .js file. There is promising progress in Closure to support 
optimizing common js and ES6 code but it is not quite there yet and can't 
compile everything. It will be coming to shadow-cljs soon though, it is 
already in CLJS but I didn't hook it up yet since I wanted to solve some 
issues first. 

:npm-module tries to bridge the gap a little but as soon as you don't 
optimize your whole program you need externs to teach Closure about the 
things it can't see. Which means you can still use :advanced compilation 
for the CLJS/Closure parts but the rest of the code remains untouched. It 
also means that you need a second bundler (eg. webpack) to then combine the 
Closure output with the remaining unoptimized parts.

Given all these constraints I don't think it is feasible to publish 
compiled (but unbundled) CLJS code as a library to NPM. It would be 
relatively easy to consume uncompiled .cljs code from NPM packages but 
unless everybody agrees to do that it would just make things a whole lot 
more complicated for everyone. Maven is also a whole lot better than NPM 
IMHO, npm or yarn just have better UX (until something breaks).

:npm-deps is a feature in CLJS that lets you declare which npm packages 
your CLJS code requires to work. Technically this would need a complement 
in package.json to allow declaring which CLJS packages the JS code needs. I 
don't have a solution for that and I was and still am opposed to whole idea 
of :npm-deps. It does however solve the issue from the CLJS perspective, 
just not from the "Play as a Team" JS perspective.

I do want integration to be painless and I do think shadow-cljs already 
makes life easier for some. I am very interested to hear about any other 
pain points people may have with CLJS<->JS interop. I tested webpack, 
create-react-app and create-react-native-app. They are all work reasonably 
well but I haven't gone beyond basic examples.

I really don't have a good solution for the problem since part of the 
solution must come from the JS world and I have basically no idea what 
people are doing over there.

The whole problem basically boils down to which bundler you want to use.

I'm personally only considering the Closure Compiler for that and 
shadow-cljs is trying to make that as easy as possible with support for 
common JS / ES6 coming soon.

If you want to use webpack or so that also works reasonably well already 
but CLJS really is built with the Closure Compiler in mind so the end 
result of this won't be optimal.

If you want both that is always going to require more manual work for 
:externs and come with certain caveats.

I hope that made some sort of sense, I have been thinking about this a lot 
but haven't gotten very far.

/thomas

On Sunday, July 9, 2017 at 3:29:15 AM UTC+2, Kurt Harriger wrote:
>
> Thomas,
>
> First thank you for starting this project!
>
> I recently joined a team at Atlassian that uses ClojureScript to support 
> collaborative editing.  I don't think I need to sell the community here on 
> the advantages of ClojureScript, but I think packaging still needs work.
>
> If you use ClojureScript for your entire UI you probably don't have any 
> issues, but in larger organizations we need to "Play as a Team." It does 
> not seem feasible or reasonable to expect that other teams switch to using 
> the ClosureScript compiler and make their js libraries play nice with 
> closure because the end result will be marginally better, especially given 
> the work involved in ensuring they still work with advanced optimizations 
> may be non-trivial.
>
> The only practical alternative I'm currently aware of is to precompile the 
> ClojureScript with optimizations and have them include this as a script tag 
> in their page.  The disadvantages of this 

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Kurt Harriger]

I understand it is important since only a small portion of the relatively
large closure code base is used in any given app... but isn't this part of
what the uglify plugin and/or webpack-closure-compiler does? Are you saying
that dead code elimination will not work as well after the code has been
commonjs-ified? Perhaps it will still work well enough?



On Sun, Jul 9, 2017 at 1:34 PM Dustin Getz  wrote:

> Closure is also doing dead code elimination which is only possible if you
> code in a restricted dialect of javascript, which cljs compiler does emit.
> Dead code elimination is important because, for example, the standard
> library and data structures are not implemented naively and bundled with
> the js vm, the standard lib has to be shipped on the wire and parsed.
>
> --
> Note that posts from new members are moderated - please be patient with
> your first post.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "ClojureScript" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojurescript/AGXku7Ous0Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojurescript+unsubscr...@googlegroups.com.
> To post to this group, send email to clojurescript@googlegroups.com.
> Visit this group at https://groups.google.com/group/clojurescript.
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Kurt Harriger]

Thomas,

First thank you for starting this project!

I recently joined a team at Atlassian that uses ClojureScript to support 
collaborative editing.  I don't think I need to sell the community here on 
the advantages of ClojureScript, but I think packaging still needs work.

If you use ClojureScript for your entire UI you probably don't have any 
issues, but in larger organizations we need to "Play as a Team." It does 
not seem feasible or reasonable to expect that other teams switch to using 
the ClosureScript compiler and make their js libraries play nice with 
closure because the end result will be marginally better, especially given 
the work involved in ensuring they still work with advanced optimizations 
may be non-trivial.

The only practical alternative I'm currently aware of is to precompile the 
ClojureScript with optimizations and have them include this as a script tag 
in their page.  The disadvantages of this are many:

* separate download - not bundled with other js libraries
* not commonjs / cannot require / pollutes global namespace 
* install/upgrade/versioning process is different / change script tag 
rather than npm/yarn
* each precompiled ClosureScript library will include closure and cljs again

As we add support for multiple editors I want to split the library into 
multiple artifacts that can be released separately, however that last 
bullet point is undesirable.  I have had some success in adding a cljsbuild 
along side a webpack build, but CommonJS has solved a lot of pain points 
for JS devs and most modern js devs now expect an npm artifact. 

Although shadow-cljs doesn't solve having different install/version process 
it does appear to solve the other issues and is better than using a script 
tag.

I did have a general question though. I understand that if you compile 
multiple ClosureScript libraries independently each will end up with 
duplicate bits of closure and cljs core. Shadow-cljs, as I understand it, 
appears to solve this by first compiling all the ClosureScript code 
together and exposing it as a commonjs module. But thinking out-loud 
here...isn't sharing dependencies the problem npm is trying to solve? could 
we leverage that instead of reinvent it?  

What if google closure libraries and cljs.core were published as their own 
commonjs npm libraries - precompiled but not minified yet? Couldn't cljs 
libraries then just 'require' them without resulting in another copy or 
requiring the closurescirpt compiler (and jvm)?  In my ideal world, I think 
I would be able to specify :npm-module as the clojurescript compiler target 
and it would generate js that can be packaged in npm that 'requires' rather 
than bundles cljs and closure.  Minification can be done downstream with 
*all* the javascript, not just the cljs code. I also found a closure plugin 
for webpack https://www.npmjs.com/package/webpack-closure-compiler (I never 
used it though), so maybe we can still get just as good if not better 
results doing it later? 

I'm not sure what would be involved in rewriting cljs output to require 
"require" closure and cljs.core as their own libraries, but maybe the 
partially solved already with 
https://github.com/clojure/clojurescript/wiki/JavaScript-Module-Support-(Alpha)?

Thanks!

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Thomas Heller]

I made a quick video showing how you'd use what we have talked about here 
with "create-react-app".

https://www.youtube.com/watch?v=BLDX5Twt2zk


-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[jiyinyiyong]

Yes Webpack will replace the dependent files. Webpack made a concept called
"module tree" based on dependency tree. File changes will propagate along
the tree, and hit the root at last. But if a change propagates to root, the
whole page reloads. That's not what we wanted.

I don't have an example, but you may follow the docs:
https://webpack.js.org/guides/hmr-react/#app-code

roughly 3 steps(I did that before but not sure about every detail)

1) add configs in `webpack.config.js`



  entry: [
'react-hot-loader/patch',
// activate HMR for React

'webpack-dev-server/client?http://localhost:8080', < add this
// bundle the client for webpack-dev-server
// and connect to the provided endpoint

'webpack/hot/only-dev-server', < add this
// bundle the client for hot reloading
// only- means to only hot reload for successful updates

'./index.js'
// the entry point of our app
  ],

  plugins: [
new webpack.HotModuleReplacementPlugin(),  < add this
// enable HMR globally

new webpack.NamedModulesPlugin(),
// prints more readable module names in the browser console on HMR updates
  ],

2) add `module.hot.accept('./demo', cb)` to handle the changes from
'./demo', so it will not reload the page:

// Hot Module Replacement APIif (module.hot) {
  module.hot.accept('./components/App', () => {
render(App)
  });}

3) replace `webpack` to `webpack-dev-server` to start bunding:

webpack-dev-server


Webpack would inject code that starts a WebSocket and responds to file
changes. Since it contains many steps, it's highly possible you run into
problems setting up Webpack HMR.

On Wed, May 17, 2017 at 6:54 PM Thomas Heller  wrote:

> The shadow-cljs compiler already does the correct thing when it comes to
> macros and recompiles all affected CLJS files.
>
> It does however know nothing about JS files that may be affected as well
> so it would not reload those. I don't know if the webpack HMR will reload
> dependent files?
>
> Does it reload ./index.js if that does require("shadow.cljs/some.foo") and
> some.foo was recompiled?
>
> I have never even seen a HMR config, can you share one so I can try a few
> things?
>
> On Wednesday, May 17, 2017 at 12:48:18 PM UTC+2, Jiyin Yiyong wrote:
> > Just realized that ClojureScript is different from CoffeeScript because
> of the macro system. If a macro changed, it may cause changes of multiple
> files. So that it's hard to just detect which file changed and compiled it
> alone. The only window left is to read and compare file content before
> writing, if we are trying to get rid of the unnecessary change events...
> >
>
> --
> Note that posts from new members are moderated - please be patient with
> your first post.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "ClojureScript" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojurescript/AGXku7Ous0Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojurescript+unsubscr...@googlegroups.com.
> To post to this group, send email to clojurescript@googlegroups.com.
> Visit this group at https://groups.google.com/group/clojurescript.
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Thomas Heller]

The shadow-cljs compiler already does the correct thing when it comes to macros 
and recompiles all affected CLJS files.

It does however know nothing about JS files that may be affected as well so it 
would not reload those. I don't know if the webpack HMR will reload dependent 
files?

Does it reload ./index.js if that does require("shadow.cljs/some.foo") and 
some.foo was recompiled?

I have never even seen a HMR config, can you share one so I can try a few 
things?

On Wednesday, May 17, 2017 at 12:48:18 PM UTC+2, Jiyin Yiyong wrote:
> Just realized that ClojureScript is different from CoffeeScript because of 
> the macro system. If a macro changed, it may cause changes of multiple files. 
> So that it's hard to just detect which file changed and compiled it alone. 
> The only window left is to read and compare file content before writing, if 
> we are trying to get rid of the unnecessary change events...
> 

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[jiyinyiyong]

Just realized that ClojureScript is different from CoffeeScript because of
the macro system. If a macro changed, it may cause changes of multiple
files. So that it's hard to just detect which file changed and compiled it
alone. The only window left is to read and compare file content before
writing, if we are trying to get rid of the unnecessary change events...

On Wed, May 17, 2017 at 6:18 PM Thomas Heller  wrote:

>
> > * in `node_modules/` there are always tens or even thousands of folders,
> which I'm never happy to open it.
> >
>
> That is indeed scary, but you'd never need to open it. Same way you never
> look at any other node_modules folder probably.
>
> >
> > * for hot module replacement issues, sometimes we want to ignore the
> whole `node_modules/` because of the amount of the files,
> https://webpack.js.org/configuration/watch/#watchoptions-ignored . I
> don't think all of us will ignore that folder, but we might do and we
> consider code in `node_modules/` remaining the same.
> >
>
> Given that you can specify a regexp you can just use
>
> ignored: /node_modules\/(?!shadow-cljs)/
>
> where it still ignores any node_modules folder except shadow-cljs.
> Watching this folder however is currently not the best idea because I was
> lazy and didn't optimize for it yet. That means that the compiler will emit
> all files on compile, not just the files that were actually
> recompiled/changed.
>
> I will fix that eventually.
> https://github.com/thheller/shadow-cljs/issues/9
>
> --
> Note that posts from new members are moderated - please be patient with
> your first post.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "ClojureScript" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojurescript/AGXku7Ous0Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojurescript+unsubscr...@googlegroups.com.
> To post to this group, send email to clojurescript@googlegroups.com.
> Visit this group at https://groups.google.com/group/clojurescript.
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[jiyinyiyong]

> Where is the difference in putting code into node_modules or compiled
then using webpack?

I'm not sure either. But here's what I see:

* in `node_modules/` there are always tens or even thousands of folders,
which I'm never happy to open it.

* for hot module replacement issues, sometimes we want to ignore the whole
`node_modules/` because of the amount of the files,
https://webpack.js.org/configuration/watch/#watchoptions-ignored . I don't
think all of us will ignore that folder, but we might do and we consider
code in `node_modules/` remaining the same.

As a result, using `resolve.alias` appears more acceptable to me.

On Wed, May 17, 2017 at 2:12 PM Thomas Heller  wrote:

> On Wednesday, May 17, 2017 at 3:51:14 AM UTC+2, Jiyin Yiyong wrote:
> > I don't like the way of writing `x.assoc(null, 'foo', 'bar')` by myself.
> Also I don't people would accept that.
> >
>
>
> I wouldn't as well, but that was an example to show that you can use CLJS
> code directly without modifying it in any way.
>
> import { assoc } from "shadow-cljs/cljs.core";
> assoc(null, "foo", "bar");
>
> also works. assoc is an ugly example because of the leading null but it is
> the shortest form to replace the CLJS {} empty map literal. {} would mean
> empty JS object which doesn't work with assoc.
>
> > I see the problem now. ClojureScript compilers may have tricky behaviors
> that make it different from CoffeeScript's compiling processes.
> >
>
> The compiler is pretty straightforward actually, the tricky parts are the
> version conflicts. If there was a way to ensure that everything always used
> the exact some compiler configuration/version this would be possible, but
> there is not AFAICT.
>
> > Putting `x.cljs` files in npm is fine, it's just slower. To me compiling
> ClojureScript is always slower, I already accepted that.
> >
>
> [:script] Build completed. (23 files, 1 compiled, 0 warnings, 0.12s)
>
> I think 0.12s is pretty fast. That is running in --dev mode recompiling
> one changed file with hot-reload enabled.
>
> Of course it is slower if you always run "shadow-cljs --once" which
> includes starting the JVM each time. "shadow-cljs --dev" will be
> substantially faster when re-compiling.
>
>
>
> >
> > I think here are may main concerns:
> >
> >
> > * in js projects, we regard code in `node_modules` are modules instead
> of source code. For web projects using Webpack, I would prefer using a
> 'compiled/' folder to hold the code and then config `resolve.alias` to make
> sure I import the code with `require('cljs/foo.bar.core')`.
> https://webpack.js.org/configuration/resolve/
> >
>
> Not sure what you mean. Where is the difference in putting code into
> node_modules or compiled then using webpack? The behavior is identical, the
> node_modules version just doesn't need the resolve.alias? Both versions
> require that you have your code compiled before running webpack, which
> directory you load them from should be automatic?
>
>
> > *  For nodejs projects, somehow I can accept the solution that we use
> `require('../../compiled/foo.bar.core')`. But I guess it may bring
> problems, which makes your `shadow-cljs` solution a better choice...
> >
>
> Yes, I hate relative paths. Actually the output folder where things end up
> is a config option. So if you really wanted to have your compiled folder
> that can be done. I just don't see an upside to it.
>
> >
> > * does shadow-devtools compiled cljs files incrementally? If it does,
> Webpack may use it to hot replace module.
> >
>
> Yes, incrementally. Don't know anything about HMR but shadow-devtools
> already has hot-reloading (ala figwheel) built-in. I just didn't enable it
> yet because other things were more important.
>
> Can't tell if Webpack HMR would work since I have never used it.
>
> Thanks for the feedback, I hope I lessened your concerns.
>
> Keep it coming.
>
> Remember: this is evolving as we discuss it, I have no idea what JS devs
> want so without feedback I'll be building what I want. That may or may not
> align.
>
> --
> Note that posts from new members are moderated - please be patient with
> your first post.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "ClojureScript" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojurescript/AGXku7Ous0Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojurescript+unsubscr...@googlegroups.com.
> To post to this group, send email to clojurescript@googlegroups.com.
> Visit this group at https://groups.google.com/group/clojurescript.
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send 

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Thomas Heller]

On Wednesday, May 17, 2017 at 3:51:14 AM UTC+2, Jiyin Yiyong wrote:
> I don't like the way of writing `x.assoc(null, 'foo', 'bar')` by myself. Also 
> I don't people would accept that.
> 


I wouldn't as well, but that was an example to show that you can use CLJS code 
directly without modifying it in any way.

import { assoc } from "shadow-cljs/cljs.core";
assoc(null, "foo", "bar");

also works. assoc is an ugly example because of the leading null but it is the 
shortest form to replace the CLJS {} empty map literal. {} would mean empty JS 
object which doesn't work with assoc.

> I see the problem now. ClojureScript compilers may have tricky behaviors that 
> make it different from CoffeeScript's compiling processes.
> 

The compiler is pretty straightforward actually, the tricky parts are the 
version conflicts. If there was a way to ensure that everything always used the 
exact some compiler configuration/version this would be possible, but there is 
not AFAICT.

> Putting `x.cljs` files in npm is fine, it's just slower. To me compiling 
> ClojureScript is always slower, I already accepted that.
> 

[:script] Build completed. (23 files, 1 compiled, 0 warnings, 0.12s)

I think 0.12s is pretty fast. That is running in --dev mode recompiling one 
changed file with hot-reload enabled.

Of course it is slower if you always run "shadow-cljs --once" which includes 
starting the JVM each time. "shadow-cljs --dev" will be substantially faster 
when re-compiling.



> 
> I think here are may main concerns:
> 
> 
> * in js projects, we regard code in `node_modules` are modules instead of 
> source code. For web projects using Webpack, I would prefer using a 
> 'compiled/' folder to hold the code and then config `resolve.alias` to make 
> sure I import the code with `require('cljs/foo.bar.core')`. 
> https://webpack.js.org/configuration/resolve/
> 

Not sure what you mean. Where is the difference in putting code into 
node_modules or compiled then using webpack? The behavior is identical, the 
node_modules version just doesn't need the resolve.alias? Both versions require 
that you have your code compiled before running webpack, which directory you 
load them from should be automatic?


> *  For nodejs projects, somehow I can accept the solution that we use 
> `require('../../compiled/foo.bar.core')`. But I guess it may bring problems, 
> which makes your `shadow-cljs` solution a better choice...
> 

Yes, I hate relative paths. Actually the output folder where things end up is a 
config option. So if you really wanted to have your compiled folder that can be 
done. I just don't see an upside to it.

> 
> * does shadow-devtools compiled cljs files incrementally? If it does, Webpack 
> may use it to hot replace module.
> 

Yes, incrementally. Don't know anything about HMR but shadow-devtools already 
has hot-reloading (ala figwheel) built-in. I just didn't enable it yet because 
other things were more important.

Can't tell if Webpack HMR would work since I have never used it.

Thanks for the feedback, I hope I lessened your concerns.

Keep it coming.

Remember: this is evolving as we discuss it, I have no idea what JS devs want 
so without feedback I'll be building what I want. That may or may not align.

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[jiyinyiyong]

I don't like the way of writing `x.assoc(null, 'foo', 'bar')` by myself.
Also I don't people would accept that.

I see the problem now. ClojureScript compilers may have tricky behaviors
that make it different from CoffeeScript's compiling processes.

Putting `x.cljs` files in npm is fine, it's just slower. To me compiling
ClojureScript is always slower, I already accepted that.

I think here are may main concerns:

* in js projects, we regard code in `node_modules` are modules instead of
source code. For web projects using Webpack, I would prefer using a
'compiled/' folder to hold the code and then config `resolve.alias` to make
sure I import the code with `require('cljs/foo.bar.core')`.
https://webpack.js.org/configuration/resolve/

*  For nodejs projects, somehow I can accept the solution that we use
`require('../../compiled/foo.bar.core')`. But I guess it may bring
problems, which makes your `shadow-cljs` solution a better choice...

* does shadow-devtools compiled cljs files incrementally? If it does,
Webpack may use it to hot replace module.

On Wed, May 17, 2017 at 2:56 AM Thomas Heller  wrote:

> On Tuesday, May 16, 2017 at 8:27:44 PM UTC+2, Thomas Heller wrote:
> > If I understand correctly the short answer is: no.
> >
>
> I should add: You can use shadow-cljs in your coffeescript project if you
> just want to use cljs.core or some other CLJS package. You do not need to
> write CLJS yourself to make use of shadow-cljs.
>
> var x = require("shadow-cljs/cljs.core");
> x.assoc(null, "foo", "bar");
>
> is basically the same as
>
> var x = require("mori")
> x.assoc(null, "foo", "bar");
>
> But you would need to run shadow-cljs in your project.
>
> --
> Note that posts from new members are moderated - please be patient with
> your first post.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "ClojureScript" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojurescript/AGXku7Ous0Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojurescript+unsubscr...@googlegroups.com.
> To post to this group, send email to clojurescript@googlegroups.com.
> Visit this group at https://groups.google.com/group/clojurescript.
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Thomas Heller]

On Tuesday, May 16, 2017 at 8:27:44 PM UTC+2, Thomas Heller wrote:
> If I understand correctly the short answer is: no.
> 

I should add: You can use shadow-cljs in your coffeescript project if you just 
want to use cljs.core or some other CLJS package. You do not need to write CLJS 
yourself to make use of shadow-cljs.

var x = require("shadow-cljs/cljs.core");
x.assoc(null, "foo", "bar");

is basically the same as

var x = require("mori")
x.assoc(null, "foo", "bar");

But you would need to run shadow-cljs in your project.

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[Thomas Heller]

If I understand correctly the short answer is: no.

I chose this setup specifically to avoid anyone ever putting compiled CLJS code 
in npm packages. The problem with this would be that everyone would bring their 
own cljs.core version again.

That is an issue currently when you write a npm package in CLJS and publish to 
npm. Each package contains its own conflicting version of cljs.core (and maybe 
others) even if you compile with :advanced.

All CLJS sources must be compiled with the same CLJS compiler, you cannot use a 
precompiled "some-package/cljs.core.js" as there is no way to guarantee that 
everyone was using the same version and so on. The way npm resolves version 
conflicts is by creating nested node_modules folders.

./node_modules/A
./node_modules/shadow-cljs (v1.0)
./node_modules/B
./node_modules/B/node_modules/shadow-cljs (v1.1)
and so on.

There is no way to ensure that B uses the same shadow-cljs version as A. By not 
having any compiled CLJS code in any npm package we can sort of get away with 
it though.

See:
https://docs.npmjs.com/how-npm-works/npm3-nondet
https://docs.npmjs.com/how-npm-works/npm3-dupe

yarn fixes some of these issues but not all.

I do however think that it would be relatively straightforward to put 
uncompiled CLJS code into npm packages and have them picked up by shadow-cljs.

So if you have

./fancy-package/src-cljs/fancy/foo.cljs

You could just require("shadow-cljs/fancy.foo") and (:require [fancy.foo]) to 
use that. While you can still require("fancy-package") normally if there also 
was JS code but never require("fancy-package/fancy.foo").

It's a bit confusing but I repeat: there must never be compiled CLJS code in a 
npm package, only uncompiled .cljs files just like we do in CLJS .jar files 
published to clojars (or maven). For now I do recommend using the normal CLJS 
methods to publish CLJS libs. Publishing to npm would be something only 
currently supported by shadow-cljs and that would not be a good idea (for now). 
Things need to be ironed out first.


I cannot explain this very well, as I am still very confused about it myself.

Sorry if you wanted something completely different.


On Tuesday, May 16, 2017 at 7:54:41 PM UTC+2, Jiyin Yiyong wrote:
> Fascinating.. it's using `node_modules/shadow-cljs` to do the trick. I can 
> see ClojureScript files are compiled into CommonJS compatible form...
> 
> 
> Do you think it's possible that we put all js built from `cjs.core`(also with 
> `goog`) into a single npm package. And then for each ClojureScript file, it 
> is mapped to a JavaScript file by `shadow-cljs` tools, which use 
> `require("./cljs.core.js");` to access it's cljs dependencies(like in your 
> demo), and use `require('package_name/foo/bar')` to require sibling files. 
> For other packages, maybe we can do the some trick, like precompile them with 
> `shadow-cljs` tools and then release them to npm. Seeing from a CoffeeScript 
> user, I think somehow in this way ClojureScript can also fit into npm's 
> ecosystem.
> 
> 
> Here's an example, we may create a new project in such file structure, where:
> 
> 
> `src/` is for source code,
> `target` is for compiled code,
> `package.json` is copied from `assets/package.json` to `target/package.json`,
> 
> 
> ```
> .
> ├── assets
> │   └── package.json
> ├── project.clj
> ├── src
> │   └── demo
> │       └── main.cljs
> └── target
>     ├── demo
>     │   └── main.js
>     └── package.json
> ```
> 
> 
> if it's in this way, `target/` can be released on npm as a module and others 
> surely may use `require('package_name/demo/main.js')` to load the package 
> built from ClojureScript.
> 
> 
> 
> 
> 
> 
> On Wed, May 17, 2017 at 12:15 AM Thomas Heller  wrote:
> 
> 
> > 1. Should we call this "cljs-require"?  With the understanding that it is a 
> > bit like a portal to cljs's ":require" form:
> 
> >
> 
> > require('cljs-require/foo.core')
> 
> 
> 
> The package name must be something you add to your package.json, otherwise 
> yarn will nuke the directory every time you run "yarn install". "npm" isn't 
> as strict but still. Using the name of the tool seemed like the obvious 
> choice. We can however create an empty cljs-require npm package that just 
> serves as a placeholder.
> 
> 
> 
> I'm totally for this if other tools want to adopt this idiom. I definitely do 
> not want this to be something you can only do with shadow-cljs.
> 
> 
> 
> 
> 
> > 2. Is there a way to set the ClojureScript version, or is it tied to 
> > shadow's version?
> 
> 
> 
> You can specify dependencies in package.json, basically like you would in 
> lein.
> 
> 
> 
> "shadow-cljs: {
> 
>     "version":"1.0.20170516",
> 
>     "dependencies": [
> 
>       ["org.clojure/clojurescript", "1.9.542"]
> 
>     ]
> 
> }
> 
> 
> 
> where "version" is the version of shadow-devtools, any other dependencies are 
> resolved using the fantastic pomegrenate lib. (thanks btw for writing the 
> dep-resolver, I didn't think it 

Re: [ClojureScript] Re: A new take on ClojureScript and npm

[jiyinyiyong]

Some typo. Since the package name in my previous post is demo, it is
actually written like `require('demo/main.js')`, which means the structure
of the folder should be(no `target/demo/`):

```
.
├── assets
│   └── package.json
├── project.clj
├── src
│   └── demo
│   └── main.cljs
└── target
├── main.js
└── package.json
```

On Wed, May 17, 2017 at 1:54 AM jiyinyiyong  wrote:

> Fascinating.. it's using `node_modules/shadow-cljs` to do the trick. I can
> see ClojureScript files are compiled into CommonJS compatible form...
>
> Do you think it's possible that we put all js built from `cjs.core`(also
> with `goog`) into a single npm package. And then for each ClojureScript
> file, it is mapped to a JavaScript file by `shadow-cljs` tools, which use
> `require("./cljs.core.js");` to access it's cljs dependencies(like in your
> demo), and use `require('package_name/foo/bar')` to require sibling files.
> For other packages, maybe we can do the some trick, like precompile them
> with `shadow-cljs` tools and then release them to npm. Seeing from a
> CoffeeScript user, I think somehow in this way ClojureScript can also fit
> into npm's ecosystem.
>
> Here's an example, we may create a new project in such file structure,
> where:
>
> `src/` is for source code,
> `target` is for compiled code,
> `package.json` is copied from `assets/package.json` to
> `target/package.json`,
>
> ```
> .
> ├── assets
> │   └── package.json
> ├── project.clj
> ├── src
> │   └── demo
> │   └── main.cljs
> └── target
> ├── demo
> │   └── main.js
> └── package.json
> ```
>
> if it's in this way, `target/` can be released on npm as a module and
> others surely may use `require('package_name/demo/main.js')` to load the
> package built from ClojureScript.
>
>
>
> On Wed, May 17, 2017 at 12:15 AM Thomas Heller 
> wrote:
>
>>
>> > 1. Should we call this "cljs-require"?  With the understanding that it
>> is a bit like a portal to cljs's ":require" form:
>> >
>> > require('cljs-require/foo.core')
>>
>> The package name must be something you add to your package.json,
>> otherwise yarn will nuke the directory every time you run "yarn install".
>> "npm" isn't as strict but still. Using the name of the tool seemed like the
>> obvious choice. We can however create an empty cljs-require npm package
>> that just serves as a placeholder.
>>
>> I'm totally for this if other tools want to adopt this idiom. I
>> definitely do not want this to be something you can only do with
>> shadow-cljs.
>>
>>
>> > 2. Is there a way to set the ClojureScript version, or is it tied to
>> shadow's version?
>>
>> You can specify dependencies in package.json, basically like you would in
>> lein.
>>
>> "shadow-cljs: {
>> "version":"1.0.20170516",
>> "dependencies": [
>>   ["org.clojure/clojurescript", "1.9.542"]
>> ]
>> }
>>
>> where "version" is the version of shadow-devtools, any other dependencies
>> are resolved using the fantastic pomegrenate lib. (thanks btw for writing
>> the dep-resolver, I didn't think it would be this simple to do the
>> dependency loading).
>>
>> I'm not sure how the conflict resolution works but you should be able to
>> use any cljs version you want. The only things written in stone are the
>> deps of the shadow-cljs uberjar, can't change the clojure version after the
>> fact.
>>
>> See:
>>
>> https://github.com/thheller/shadow-devtools/blob/master/npm-package/project.clj
>>
>>
>> >
>> > 3. Thinking back on Jiyin's question about using cljs in webpack—have
>> you put any thought into the possibility of this workflow as a webpack
>> loader?  Maybe that's a next step?
>>
>> Don't need a loader. It could be a plugin that just calls "shadow-cljs
>> --once"
>>  before proceeding with the rest of the JS compilation.
>>
>> I actually started by writing a plugin but to be honest I do not think it
>> is super useful. You can achieve the same via npm scripts & co. You'll need
>> the CLI tool for more advanced tasks (eg. REPL) anyways.
>>
>> I want this to be generally useful, not just webpack. No specific webpack
>> configuration is needed, you just somehow need to compile before calling
>> webpack. "webpack -w" and "shadow-cljs --dev" side-by-side actually just
>> works.
>>
>> > Awesome job turning this webpack discussion into a real tool we can
>> explore! 
>>
>> Thanks.
>>
>> One issue that needs to be addressed is the widespread use of CLJSJS
>> packages in popular libs like reagent Can't have cljsjs.react react.js
>> if you are supposed to use npm react.
>>
>> I can easily skip over all foreign-libs but then (:require
>> [cljsjs.react]) wouldn't have the effect of ensuring that "React" is
>> available on global.  Not yet sure how to best handle that yet.
>>
>> Still have a few ideas I want to explore, haven't totally given up on the
>> Closure Compiler yet but it making life harder than it should be.
>>
>> --
>> Note that posts from new members are moderated -