[ClojureScript] Re: Ideas on how to change my code to compile faster
Are you by any chance using anything other than {:optimizations :none} in your
dev build config? 12s seems a bit excessive if you are just doing CLJS
recompiles.
Ensure that your dev build does not use any production build settings. Your CI
server should probably run in production mode though as 12s shouldn't hurt that
much there. Typically when it comes to building larger projects you want to
rely on :none with caching for development. It would help if you could share
your build config.
If you are feeling adventurous you could try shadow-build if only to get a more
detailed report on where the time is spent. It should be a bit faster overall
as well but not by much. Happy to walk you through an example if you want to
test it.
/thomas
On Friday, February 10, 2017 at 6:15:34 PM UTC+1, Ben Brinckerhoff wrote:
> First of all, thanks to everyone for their hard work on Clojurescript and
> related tooling. It’s an incredibly productive and reliable stack to use.
>
> I’m investigating ways to speed up compile times for a closed-source project.
> We have about 8000 Clojurescript LOC and 200 Clojure LOC (in src and test
> combined). Some very rough indicators: a fresh compile of our test build
> takes about 60s. A small change to a file with maybe 15 reverse dependencies
> takes about 12s using `cljsbuild auto`. We are using lein-cljsbuild 1.1.4 and
> clojurescript “1.9.293”.
>
> These times are pretty good, but of course speeding up compiles shrinks our
> feedback loop, both locally and on CI, where we do a number of fresh compiles
> for different builds. As a result, we want to see if there are things we can
> do to our code to speed up compiles.
>
> We have turned on the `verbose` and `compiler-stats` flags so we can see more
> information about compile times. We hope to upgrade to 1.9.456 soon so we can
> see per-file compile stats. We also need to investigate parallel builds again
> - we had previously run into bugs here, but I didn’t take the time to
> investigate more fully.
>
> Besides total LOC, are there other aspects of code bases that are known to
> slow down compiles? Perhaps macro expansion (we have a lot of core.async `go`
> blocks in some namespaces)? Perhaps the complexity of the dependency graph
> between namespaces? Something else? Has anyone else had experience altering a
> CLJS code base to improve compile times? Or tweaking compiler flags?
> `optimization` makes a big difference of course, but for my current
> investigation, I'm ignoring optimization time.
>
> Also, I noticed that Clojurescript performance is an idea for the Google
> Summer of Code
> https://github.com/clojars/clojure-gsoc-2017/blob/master/project-ideas.md#clojurescript-performance
>
> “There are many impactful enhancements we would like to make to ClojureScript
> with respect to both runtime and compile time performance … For compile time
> enhancements we should examine where parallelization, AST data representation
> changes, or more aggressive caching of intermediate artifacts may deliver
> faster development and production build time.”
>
> I did a quick search of Clojurescript for perf issues on JIRA, but didn’t see
> anything related to these (apologies if I just missed something obvious!). Is
> there a list somewhere of open issues that might improve perf in the
> compiler? Or are those ideas mostly in the “needs investigation” stage?
>
> Thanks very much!
> Ben
--
Note that posts from new members are moderated - please be patient with your
first post.
---
You received this message because you are subscribed to the Google Groups
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojurescript+unsubscr...@googlegroups.com.
To post to this group, send email to clojurescript@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.
[ClojureScript] Re: Ideas on how to change my code to compile faster
Thanks very much for the ideas!
We are currently using {:optimizations :simple} for this build. All of the
times I quoted above excluded the optimization time, however (I could see the
compile time due to :verbose + :compiler-stats). We're in the process of
switching to {:optimizations :whitespace} (we were looking at :none, but we
need to generate a single file, since this is for a React Native project).
> It would help if you could share your build config.
Good point! Here is the cljsbuild build
:ci {:source-paths ["src" "src-ios" "test" "test-ios"]
:warning-handlers [foo.cljsbuild-util/fail-on-warning]
:compiler {:output-to "ios/foo/main.jsbundle"
:parallel-build false
:output-dir "target/ci/"
:optimizations :simple ; <- we're in the process of changing this to
"whitespace"
;; Rules for pulling in RN bundle
:closure-extra-annotations ["generated" "internal"]
:closure-defines {"foo.app.config.foo_host" ""
"foo.app.config.test_QMARK_" true}
:language-in :ecmascript5
:foreign-libs [{:file "../build/react-native-dev.js"
:provides ["react-native"]}
{:file "../build/react-requires-dev.js"
:provides ["react-requires"]}]
:externs ["externs.js"]
:verbose true
:compiler-stats true}}
and our lein profile is
:ci {:env {:test-multiplier "0.5"} ; environ variable
:jvm-opts ^:replace ["-Xms1024m" "-Xmx1024m" "-server" "-Xss2m"]
I was unaware of shadow-build - thanks for the idea! I'll take a look.
Ben
On Saturday, February 11, 2017 at 1:26:36 AM UTC-7, Thomas Heller wrote:
> Are you by any chance using anything other than {:optimizations :none} in
> your dev build config? 12s seems a bit excessive if you are just doing CLJS
> recompiles.
>
> Ensure that your dev build does not use any production build settings. Your
> CI server should probably run in production mode though as 12s shouldn't hurt
> that much there. Typically when it comes to building larger projects you want
> to rely on :none with caching for development. It would help if you could
> share your build config.
>
>
> If you are feeling adventurous you could try shadow-build if only to get a
> more detailed report on where the time is spent. It should be a bit faster
> overall as well but not by much. Happy to walk you through an example if you
> want to test it.
>
> /thomas
>
> On Friday, February 10, 2017 at 6:15:34 PM UTC+1, Ben Brinckerhoff wrote:
> > First of all, thanks to everyone for their hard work on Clojurescript and
> > related tooling. It’s an incredibly productive and reliable stack to use.
> >
> > I’m investigating ways to speed up compile times for a closed-source
> > project. We have about 8000 Clojurescript LOC and 200 Clojure LOC (in src
> > and test combined). Some very rough indicators: a fresh compile of our test
> > build takes about 60s. A small change to a file with maybe 15 reverse
> > dependencies takes about 12s using `cljsbuild auto`. We are using
> > lein-cljsbuild 1.1.4 and clojurescript “1.9.293”.
> >
> > These times are pretty good, but of course speeding up compiles shrinks our
> > feedback loop, both locally and on CI, where we do a number of fresh
> > compiles for different builds. As a result, we want to see if there are
> > things we can do to our code to speed up compiles.
> >
> > We have turned on the `verbose` and `compiler-stats` flags so we can see
> > more information about compile times. We hope to upgrade to 1.9.456 soon so
> > we can see per-file compile stats. We also need to investigate parallel
> > builds again - we had previously run into bugs here, but I didn’t take the
> > time to investigate more fully.
> >
> > Besides total LOC, are there other aspects of code bases that are known to
> > slow down compiles? Perhaps macro expansion (we have a lot of core.async
> > `go` blocks in some namespaces)? Perhaps the complexity of the dependency
> > graph between namespaces? Something else? Has anyone else had experience
> > altering a CLJS code base to improve compile times? Or tweaking compiler
> > flags? `optimization` makes a big difference of course, but for my current
> > investigation, I'm ignoring optimization time.
> >
> > Also, I noticed that Clojurescript performance is an idea for the Google
> > Summer of Code
> > https://github.com/clojars/clojure-gsoc-2017/blob/master/project-ideas.md#clojurescript-performance
> >
> > “There are many impactful enhancements we would like to make to
> > ClojureScript with respect to both runtime and compile time performance …
> > For compile time enhancements we should examine where parallelization, AST
> > data representation changes, or more aggressive caching of intermediate
> > artifacts may deliver faster development and production build time.”
> >
> > I did a quick search
[ClojureScript] Re: Ideas on how to change my code to compile faster
You should try adding {:cache-analysis true} to your :ci target. Caching is
otherwise only enabled for :none. Maybe that helps.
I have not looked at react-native much yet, not sure what special treatment JS
files need to run in there. shadow-build probably won't help you much in this
instance.
/thomas
On Saturday, February 11, 2017 at 4:24:01 PM UTC+1, Ben Brinckerhoff wrote:
> Thanks very much for the ideas!
>
> We are currently using {:optimizations :simple} for this build. All of the
> times I quoted above excluded the optimization time, however (I could see the
> compile time due to :verbose + :compiler-stats). We're in the process of
> switching to {:optimizations :whitespace} (we were looking at :none, but we
> need to generate a single file, since this is for a React Native project).
>
> > It would help if you could share your build config.
>
> Good point! Here is the cljsbuild build
>
> :ci {:source-paths ["src" "src-ios" "test" "test-ios"]
>:warning-handlers [foo.cljsbuild-util/fail-on-warning]
>:compiler {:output-to "ios/foo/main.jsbundle"
>:parallel-build false
>:output-dir "target/ci/"
>:optimizations :simple ; <- we're in the process of changing this to
> "whitespace"
> ;; Rules for pulling in RN bundle
> :closure-extra-annotations ["generated" "internal"]
>:closure-defines {"foo.app.config.foo_host" ""
> "foo.app.config.test_QMARK_" true}
>:language-in :ecmascript5
>:foreign-libs [{:file "../build/react-native-dev.js"
>:provides ["react-native"]}
> {:file "../build/react-requires-dev.js"
>:provides ["react-requires"]}]
> :externs ["externs.js"]
> :verbose true
>:compiler-stats true}}
>
> and our lein profile is
>
> :ci {:env {:test-multiplier "0.5"} ; environ variable
> :jvm-opts ^:replace ["-Xms1024m" "-Xmx1024m" "-server" "-Xss2m"]
>
> I was unaware of shadow-build - thanks for the idea! I'll take a look.
>
> Ben
>
> On Saturday, February 11, 2017 at 1:26:36 AM UTC-7, Thomas Heller wrote:
> > Are you by any chance using anything other than {:optimizations :none} in
> > your dev build config? 12s seems a bit excessive if you are just doing CLJS
> > recompiles.
> >
> > Ensure that your dev build does not use any production build settings. Your
> > CI server should probably run in production mode though as 12s shouldn't
> > hurt that much there. Typically when it comes to building larger projects
> > you want to rely on :none with caching for development. It would help if
> > you could share your build config.
> >
> >
> > If you are feeling adventurous you could try shadow-build if only to get a
> > more detailed report on where the time is spent. It should be a bit faster
> > overall as well but not by much. Happy to walk you through an example if
> > you want to test it.
> >
> > /thomas
> >
> > On Friday, February 10, 2017 at 6:15:34 PM UTC+1, Ben Brinckerhoff wrote:
> > > First of all, thanks to everyone for their hard work on Clojurescript and
> > > related tooling. It’s an incredibly productive and reliable stack to use.
> > >
> > > I’m investigating ways to speed up compile times for a closed-source
> > > project. We have about 8000 Clojurescript LOC and 200 Clojure LOC (in src
> > > and test combined). Some very rough indicators: a fresh compile of our
> > > test build takes about 60s. A small change to a file with maybe 15
> > > reverse dependencies takes about 12s using `cljsbuild auto`. We are using
> > > lein-cljsbuild 1.1.4 and clojurescript “1.9.293”.
> > >
> > > These times are pretty good, but of course speeding up compiles shrinks
> > > our feedback loop, both locally and on CI, where we do a number of fresh
> > > compiles for different builds. As a result, we want to see if there are
> > > things we can do to our code to speed up compiles.
> > >
> > > We have turned on the `verbose` and `compiler-stats` flags so we can see
> > > more information about compile times. We hope to upgrade to 1.9.456 soon
> > > so we can see per-file compile stats. We also need to investigate
> > > parallel builds again - we had previously run into bugs here, but I
> > > didn’t take the time to investigate more fully.
> > >
> > > Besides total LOC, are there other aspects of code bases that are known
> > > to slow down compiles? Perhaps macro expansion (we have a lot of
> > > core.async `go` blocks in some namespaces)? Perhaps the complexity of the
> > > dependency graph between namespaces? Something else? Has anyone else had
> > > experience altering a CLJS code base to improve compile times? Or
> > > tweaking compiler flags? `optimization` makes a big difference of course,
> > > but for my current investigation, I'm ignoring optimization time.
> > >
> > > Also, I noticed that Clojurescript performa
