Hi Paul! Thanks for chiming in. I set out to create a suite of middlewares for frontend optimization. The first was Catenate, which concerned itself with concatenation into bundles. So I certainly agree with your points. You'd be hard pressed to think otherwise in the Clojure community, I think, with its focus on "decomplecting". The reason I gave up on that idea is two-fold:
- The different optimizations are not orthogonal. - Assets aren't first class in the Ring middleware stack. Some examples: - When you bundle files together, your HTML has to reference either the bundle URL (in prod) or all the individual files (in dev). There has to be some sort of lookup from the bundle ID to a list of URLs, and this is dependent on your asset-serving strategy. - When you add cache-busters to URLs, you need some sort of lookup from the original URL to the cache-busted URL, so you can link to them with a known name. In other words, both the bundle middleware and the cache-busting middleware either needs to own the list of assets, or it needs to rest on a first class asset concept in the stack. Now add the ability to serve WebP images to browsers that support it. Not only do you have to change the image URLs, but you also have to serve a different set of CSS to use these new images. So this middleware would have to know which CSS files reference which files, and rewrite them. All of these could be fixed with a well-thought out Asset concept in the ring middleware stack. Which is what Optimus is an attempt at. It adds a list of assets to the request, with enough information for the linking functions to figure out which versions of which files to link. But then there's the orthogonality: - You can't add cache-busters first, and then bundle assets together, since you wouldn't get cache buster URLs on your bundles. - If you minify first, then bundle, you'll get suboptimal minification results in production. If you bundle first, then minify, you won't know which file is to blame for errors in development. - You should never add far-future expires headers unless the asset has a cache-buster URL. So ordering matters. You can't just throw in another middleware, you have to order it just so-and-so. I started writing documentation for this in Catenate. It would say "If you're also using cache-busting middleware, make sure to place it after Catenate." After writing a few of those sentences, I came to the conclusion that they were not entirely separate things. Since they're so dependent on each other, they should live together. There's also the case of *when* to optimize. In production you want to optimize once - either as a build step, or when starting the application. In development you don't want any optimization (unless you're debugging), but you still need to create the list of assets so you're able to link to it. This is something all the optimization middlewares would have to tackle on their own - basically each layer freezing their optimized assets on server start, and all but the last one doing so in vain. Optimus solves this by creating a separate middleware stack for optimizations, that work on assets (not requests), and that can be done at different times by different asset-serving strategies. So, in conclusion: If we can agree on a data format for assets - the one in Optimus has been through quite a few iterations (https://github.com/magnars/optimus#what-are-these-assets-anyway-they-seem-magical-to-me) - then we could build on that to create separate middlewares. But not middlewares for the Ring stack. It would have to be Asset-specific middlewares. For instance, even tho Optimus doesn't do transpiling, building a transpiler to fit in the Optimus asset middleware stack is pretty nice. You let :original-url be the original "styles.less", so the linking features can find it, replace the :contents with CSS, and serve it under the :path "styles.css". If your package takes a list of assets, and returns a list of assets with all .less files changed like this, you can plug it in with no modifications to Optimus. - Magnar On Tuesday, November 26, 2013 3:12:28 AM UTC+1, Paul Biggar wrote: > > [FYI: I'm the author/forker of stefon] > > These libraries aren't bad, but to be honest, I think we've done it all > wrong. We're just all scratching our own itches, not writing reusable > components (unlike most of the rest of the clojure web ecosystem). > > If you look at stefon, you get a set of non-composable functions, designed > for my exact use case. Optimus, cornet, and dieter all suffer from the same > problems. > > Cornet, supports compilation (with different compiler and versions from > stefon), serving assets, and configuring itself from a JVM command line. It > also has :dev and :prod modes (though these mean "dont minify" and "do > minify". It does split out the functions in a slightly composable way (you > can have `(wrap-lesscss-processor loader :mode :dev)`), but those still use > :mode. > > Stefon supports: concatenating JS and CSS, compiling less, coffeescript, > hamlcoffee, minification using Closure compiler (currently broken), some > trivial css minification, cache-busting and expiry headers, and asset > references (eg data-uri to put the contents of one asset in another). It > also supports caching compilation and clj-v8 for speed, and expiration > headers and cache busting. It has precompilation for production use with a > CDN). > > Dieter is basically like stefon, but older and less maintained, and some > different choices around precompilation. > > Optimus supports concatenating, minification, cache busting, expiry > headers, and something angular specific. Also assets dont have to be files > on disk, and a there's a list of a dozen or so other features that contrast > it to stefon or cornet in a neutral way (neither decision is right or > wrong, just different). > > So basically stefon supports all high level use cases, optimus support all > of them except compilation, cornet supports most of it but not cache > busting. But where we do support the same things, we do it in many > different ways. > > So my problem here is that we've each chosen to couple everything > together. If you want to use stefon's CDN feature in production, with > optimus' dev asset serving, with cornet's minification, you're out of luck. > > Weavejester made a critique of stefon on reddit [1] a while back. He made > the point "why isn't this just middleware", by which I believe he means > "why cant this all be split up into composable functions?" > > He's right. We should, IMO, stop doing what we're doing, and rebuild our > projects into a set of composable components that play well together: > > - a choice of caching middlewares > - a choice of minifying middlewares > - a choice of asset compilers (including different versions and > implementations) (also some way for them to interact to support a pipeline) > - a choice of precompilation/CDN and compiling on the server > - composable optimizations (caching compilations) > - a choice of how to concatenate assets > > I don't really know how to do this though, just that they should be > different libraries, and that all orthogonal feature sets should be > composable. I'd love to hear people's thoughts on how we can accomplish > this. > > [1] > http://www.reddit.com/r/Clojure/comments/1n1n0p/circlecistefon_asset_pipeline_for_clojure_closely/ccexi3a<http://www.google.com/url?q=http%3A%2F%2Fwww.reddit.com%2Fr%2FClojure%2Fcomments%2F1n1n0p%2Fcirclecistefon_asset_pipeline_for_clojure_closely%2Fccexi3a&sa=D&sntz=1&usg=AFQjCNFC5zUvXm1s4TP_BrpeCbz3Sz2ucQ> > > On Monday, 25 November 2013 11:10:54 UTC-8, Magnar Sveen wrote: >> >> Hi Jason! >> >> Magnar, could you talk a little about how your project is better >>> than/different from Stefon/dieter and cornet? I feel like we have a lot of >>> these projects now, all doing mostly the same thing. >>> >> >> Thanks for asking. I'll try to shed some light on the differences as I >> see them, and hopefully the people behind Dieter/Stefon (they're very >> similar) can add some details into their thinking. I haven't seen Comet, >> and google didn't help much either. Can you share a link? >> >> As for Optimus vs Stefon: First of all it's a difference in focus. Stefon >> focuses on being an asset pipeline modelled after Sprockets in Rails. It >> lets you write LESS, CoffeeScript, Haml - turning it into CSS and >> JavaScript. Optimus is not about transpiling from other languages, but >> about frontend optimization. As such, it rewrites your urls to include >> cache busters and serves your assets with far-future expires headers, so >> they can be cached aggressively in production. As I add more features to >> optimus, they too will focus around better frontend performance - and not >> more languages to be transpiled. >> >> While this is the main point in my mind, there are also other differences >> that aren't just details that everyone will agree on. :-) These two come to >> mind: >> >> 1. Stefon serves assets live in development, but requires a build step in >> production to precompile the files. Optimus does not require a build step, >> but compiles your asset when the server starts. >> >> 2. Stefon creates bundles by having custom .stefon files with >> edn-flavored lists of files in your directories of static assets. Optimus >> also needs a list of bundles, but does so using Clojure in your program. I >> would think that Stefons' approach is better if your frontend developers >> are not comfortable editing the Clojure code, while Optimus' approach >> allows more programatic control. >> >> I hope I haven't misrepresented Stefon in any way - these are my >> impressions. Since I'm a front-end optimization nut, these arguments were >> enough to sway me to create a different package. It would be several major >> breaking changes to Dieter and Stefon's architectures and APIs, and I >> didn't think I would get anywhere fighting for these changes in github >> issues. >> >> >> >>> I also don't totally understand why they're all done as Ring middleware >>> instead of lein/maven plugins. Maybe this is my Java background talking, >>> but that seems to me to be the logical place to put this sort of thing. >>> >> >> Front-end development with a compilation step is pretty horrible. There's >> also the case that the URL to a static asset and its location on disk is >> entirely different after optimization. >> >> Hope that answers your questions somewhat decently. And if it didn't, >> maybe you'll be swayed by the argument that a little competition is a good >> thing for the community. :) >> >> - Magnar >> > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
