> Since "config/releases.exs" run only during releases, if you have a
syntax error (or similar), you will find about it just way too late
We solved this problem by setting dynamic runtime configuration ONLY in
"config/releases.exs" like so
config :my_app, base_url: URI.parse(System.fetch_env!("BASE_URL"))
And "dev.exs" sets some defaults and uses the same releases.exs to
configure applications:
[{"BASE_URL", "http://localhost"}]
|> Enum.each(fn {key, value} -> System.get_env(key) || System.put_env(key,
value) end)
import_config "releases.exs"
Of course, when used like that, it wouldn't affect the already started
applications which rely on configuration during the startup (as you mention
:kernel, :stdlib, :elixir and :mix).
However we don't import "releases.exs" in other environments:
* "prod" is only used for compilation on CI, so there's no need to load
dynamic runtime configuration, and it contains only production-grade
compile-time configuration and non-configurable runtime configuration;
* "test" contains its own set of settings completely separate from the rest
but it's possible to `import "prod.exs"` if that reduces repetitions.
So I suppose our usage of "releases.exs" is similar to what you are
proposing with "runtime.exs".
However I think using it unconditionally in every environment will only
complicate things, as you mention we'd have to branch on Mix.env/Config.env
- this seems more complex than our current approach.
> In the future we will deprecate Application.get_env/3 in the module body.
Does that mean during module compilation? I guess it'll still be available
in normal runtime?
As I understand "runtime.exs" is supposed to be simply copied to the
release.
Although it looks reasonably safe, I definitely wouldn't want to see some
testing or development settings in a file included in a release especially
if they are unused.
Maybe we should come to this problem from a different side.
What if we introduce a separate "compilation" stage (using separate set of
environments - e.g. dev_compile/prod_compile) and start a clean VM for the
real runtime after Mix compiled the applications?
Kind regards,
Dmitry Belyaev
On Thu, Mar 5, 2020 at 4:20 AM José Valim <jose.va...@dashbit.co> wrote:
> Over the last releases we have been making improvements to the
> configuration system. However, there is still one large gap to be
> addressed. Today config files are treated as compile-time configuration by
> default. This behaviour, alongside overuse from libraries, made using the
> application environment via config files more rigid and error prone than
> they have to be.
>
> Today, Mix will load "config/config.exs" before any of the dependencies
> are compiled and on every command. This makes basic scenarios like the ones
> below impossible:
>
> -
>
> If your project requires some environment variables in development,
> you cannot enforce them. Otherwise, basic commands such as mix help or
> editor integration will stop working
>
>
> -
>
> If you need to configure one application based on another application,
> that is impossible to do as well, as no applications are available at this
> point
>
> Also, because configuration files run during compilation by default,
> library authors often accidentally rely on compile-time configuration.
> Luckily, Elixir v1.10 has added Application.compile_env/2. In the future
> we will deprecate Application.get_env/3 in the module body, which will
> help steer people to the correct direction in their own applications.
>
> On the opposite side, we have releases, where the configuration works at
> runtime only. Given there is no way to execute configuration at runtime in
> Mix, we end-up with two completely disjoint approaches. This introduces a
> couple issues of their own:
>
> -
>
> Since "config/releases.exs" run only during releases, if you have a
> syntax error (or similar), you will find about it just way too late
>
>
> -
>
> There is no way for frameworks like Phoenix to provide a configuration
> file that works for both Mix and release deployments
>
> Our goal is to address these issues. However, it is important to consider
> that a complex project today may already have many configuration files:
>
> - config/config.exs (compile time) - shared configuration settings
> - config/{dev,test,prod}.exs (compile time) - per env configuration
> settings
> - config/releases.exs (runtime) - release specific configuration
> settings
>
> Therefore, we would like to propose a new configuration file that can
> address the problem above while replacing the need to use
> "config/releases.exs" in 99% of the cases.
> Proposal: introduce config/runtime.exs
>
> Our proposal is simple: we will introduce "config/runtime.exs".
> "config/runtime.exs" will be loaded both by Mix and Releases, closing the
> gap between them.
>
> For Mix, "config/runtime.exs" will load after the code is compiled and
> before the application starts, this allows "config/runtime.exs" to rely on
> code from dependencies, as long as you keep in mind that any application
> that is started during "config/runtime.exs" cannot be configured by
> "config/runtime.exs" itself. Furthermore, given "config/runtime.exs" works
> at runtime, changing it won't require the whole application to be
> recompiled.
>
> For Releases, it will work precisely the same as "config/releases.exs". If
> both are available, "config/runtime.exs" is executed first, followed by
> "config/releases.exs".
>
> There are a couple pitfalls to be aware though:
>
> -
>
> Since "config/runtime.exs" is used by both Mix and releases, it cannot
> configure :kernel, :stdlib, :elixir, and :mix themselves. Attempting
> to configure those will emit an error. For those rare scenarios, you will
> need to use "config/releases.exs" - but "config/releases.exs" will remain
> simple, which will reduce the odds of syntax errors.
>
>
> -
>
> Since "config/runtime.exs" is used by both Mix and releases, it cannot
> invoke "Mix" directly. Therefore, for conditional environment compilation,
> we will add a env/2 macro to Config that will be available for all
> config files. For example, instead of a "config/runtime.prod.exs", one will
> have to:
>
> import Config
>
> env :prod do
> config :my_app, :secret_key, System.get_env!("SECRET_KEY")end
>
>
> One may argue that "config/runtime.exs" should eventually replace
> "config/config.exs" as the default file for application configuration. We
> will certainly evaluate this option in the future but it is important to
> take baby steps. And the first step is to support "config/runtime.exs". :)
> Implementation considerations
>
> This section covers implementation details. It is not part of the proposal
> per se. Although the feature is relatively small, it requires many
> improvements to Mix and the underlying config engine. The tasks are:
>
> - Load config/runtime.exs inside Mix
> - Copy config/runtime.exs inside escripts and load them when the
> escript runs
> - Copy config/runtime.exs inside releases (similar to
> config/releases.exs)
> - Add a feature to Config.Reader that allows a warning to be emitted
> if an undesired module is used (for example, Mix)
> - Add the env/2 macro to Config
> - Raise if "import_config" is used in "config/runtime.exs" and
> "config/releases.exs" - providing proper guidance to users
>
> One aspect to consider is exactly when runtime config should be loaded
> inside Mix. We need to choose between doing it after the "compile" task or
> before "app.start". The issue is that many projects have tasks that only
> need the application to be compiled but not started. For example, Ecto Repo
> management tasks or Phoenix routes tasks. Those tasks today simply run
> Mix.Task.run("compile"). However, if we were to introduce
> "config/runtime.exs" and load it before "app.start", those tasks will now
> run without "config/runtime.exs" and behave incorrectly.
>
> Therefore there is an argument to be made to load the runtime
> configuration right after the code is compiled - even though this is a bit
> unintuitive. The other option is to ask users to always run "app.start" as
> the entry point and pass the "--no-start" if they actually don't want to
> start their apps, which is also a bit counter intuitive. Unfortunately, the
> second option means projects will behave incorrectly until they are updated.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elixir-lang-core+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BS7nefgLCXCb9OWQGj9d9eE1TrDstq%3Dkm%3D%2B2a7iT9DAw%40mail.gmail.com
> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BS7nefgLCXCb9OWQGj9d9eE1TrDstq%3Dkm%3D%2B2a7iT9DAw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/CAEY66iOe713gMzAkOLQGYv9h%2B4%3DG8T2Oa8fMAR%2B04VkAykLZfQ%40mail.gmail.com.
