[Nix-dev] Tips on deploying a Scala Play application
Hi Nix-devs, This is a plain request for assistance / best practices for using Nixos with Java / Scala / Play. Akka with EventSourcing are also a topic of interest. I'm currently trying to get a Scala Play app up and running on my nixOps deployed machines. As I'm very unfamiliar with running Java based apps, I'd like to know if someone has experience on the common pitfalls and tips on keeping the servers healthy (I just caused my laptop's 8 cores to go 100% without being able to stop the server started by the activator call). I've seen some related packages in nixpkgs and have many questions like eg. do I need sbt (which seems to provide typesafe - activator) and a jdk on the production servers or are is a jre sufficient? How do I deploy and run a Java app developed locally? And how do I set-up a local nixos vm for Java development? I'm still investigating and learning a lot myself, so nix-related knowledge is my main concern here (as I need to figure out the rest myself anyway ;-) I'll be happy to share my findings and configuration / setup.. Kind regards, Erik ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Tips on deploying a Scala Play application
Hi, A JRE should be enough for running it, but you need sbt and a JDK for building. I've got a derivation for a Play website at https://github.com/BlocklandGlass/ParseTS-Playground/blob/master/parsets-playground.nix, with the NixOS/NixOps setup at https://github.com/BlocklandGlass/ParseTS-Playground/tree/master/deployment. The gist of it is to run "sbt stage" in the build phase, and to then take "target/universal/stage" as your build output. However, you'll also need to wrap the launcher script to add your JRE and to add gawk (which the launcher script requires). Finally, on any modern system (such as NixOS) you'll also want to disable Play's PID file management, since systemd takes care of that anyway. I didn't in that script, but you'll probably also want to add a testing phase as part of the build. The big drawback with this approach is that SBT downloads all dependencies from the internet on demand, which won't work on a Nix setup with proper isolation (ideally, builds should only have network access if they deterministically produce a given hash). I've been toying with the idea of writing a sbt2nix SBT plugin that generates Nix definitions to build a local maven mirror for the dependencies, but I haven't got around to that (yet). // Teo On 5 July 2016 at 21:52, 4levels <4lev...@gmail.com> wrote: > Hi Nix-devs, > > This is a plain request for assistance / best practices for using Nixos > with Java / Scala / Play. Akka with EventSourcing are also a topic of > interest. > > I'm currently trying to get a Scala Play app up and running on my nixOps > deployed machines. As I'm very unfamiliar with running Java based apps, > I'd like to know if someone has experience on the common pitfalls and tips > on keeping the servers healthy (I just caused my laptop's 8 cores to go > 100% without being able to stop the server started by the activator call). > > I've seen some related packages in nixpkgs and have many questions like > eg. do I need sbt (which seems to provide typesafe - activator) and a jdk > on the production servers or are is a jre sufficient? How do I deploy and > run a Java app developed locally? > And how do I set-up a local nixos vm for Java development? > > I'm still investigating and learning a lot myself, so nix-related > knowledge is my main concern here (as I need to figure out the rest myself > anyway ;-) > > I'll be happy to share my findings and configuration / setup.. > > > Kind regards, > > Erik > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Tips on deploying a Scala Play application
Hi Teo, Thank you for your explanation and quick qualitative response! I'll be looking at your code asap and report back with my experiences ;-) Kind regards, Erik On Tue, Jul 5, 2016, 22:08 Teo Klestrup Röijezon wrote: > Hi, > > A JRE should be enough for running it, but you need sbt and a JDK for > building. I've got a derivation for a Play website at > https://github.com/BlocklandGlass/ParseTS-Playground/blob/master/parsets-playground.nix, > with the NixOS/NixOps setup at > https://github.com/BlocklandGlass/ParseTS-Playground/tree/master/deployment > . > > The gist of it is to run "sbt stage" in the build phase, and to then take > "target/universal/stage" as your build output. However, you'll also need to > wrap the launcher script to add your JRE and to add gawk (which the > launcher script requires). Finally, on any modern system (such as NixOS) > you'll also want to disable Play's PID file management, since systemd takes > care of that anyway. I didn't in that script, but you'll probably also want > to add a testing phase as part of the build. > > The big drawback with this approach is that SBT downloads all dependencies > from the internet on demand, which won't work on a Nix setup with proper > isolation (ideally, builds should only have network access if they > deterministically produce a given hash). > > I've been toying with the idea of writing a sbt2nix SBT plugin that > generates Nix definitions to build a local maven mirror for the > dependencies, but I haven't got around to that (yet). > > // Teo > > On 5 July 2016 at 21:52, 4levels <4lev...@gmail.com> wrote: > >> Hi Nix-devs, >> >> This is a plain request for assistance / best practices for using Nixos >> with Java / Scala / Play. Akka with EventSourcing are also a topic of >> interest. >> >> I'm currently trying to get a Scala Play app up and running on my nixOps >> deployed machines. As I'm very unfamiliar with running Java based apps, >> I'd like to know if someone has experience on the common pitfalls and tips >> on keeping the servers healthy (I just caused my laptop's 8 cores to go >> 100% without being able to stop the server started by the activator call). >> >> I've seen some related packages in nixpkgs and have many questions like >> eg. do I need sbt (which seems to provide typesafe - activator) and a jdk >> on the production servers or are is a jre sufficient? How do I deploy and >> run a Java app developed locally? >> And how do I set-up a local nixos vm for Java development? >> >> I'm still investigating and learning a lot myself, so nix-related >> knowledge is my main concern here (as I need to figure out the rest myself >> anyway ;-) >> >> I'll be happy to share my findings and configuration / setup.. >> >> >> Kind regards, >> >> Erik >> >> ___ >> nix-dev mailing list >> nix-dev@lists.science.uu.nl >> http://lists.science.uu.nl/mailman/listinfo/nix-dev >> >> > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Tips on deploying a Scala Play application
Hi Teo, I've come quite far in setting up things, but I keep running into building errors. It has everything to do with me removing all references to parsets and postgres and renaming things here and there, trying to merge them with the current deploy setup. Do I understand correctly that parsets is a library to store data, using postgres in the background? I'd like to start using Event Sourcing with Scala / Akka so I don't need a datastore like parsets, correct? I'm very unsure about this as I literally started today with learning Scala / Play. I got my toes wet with Java before but that's really it. Something else I found interesting as I'm quite an Nginx fan and have nginx running with proxies already: Nginx has capabilities to deal with Java in different ways, as proxy or tied with eg Clojure for even faster results.. The journey continues ;-) Kind regards, Erik On Tue, Jul 5, 2016 at 10:23 PM 4levels <4lev...@gmail.com> wrote: > Hi Teo, > > Thank you for your explanation and quick qualitative response! > > I'll be looking at your code asap and report back with my experiences ;-) > > Kind regards, > > Erik > > On Tue, Jul 5, 2016, 22:08 Teo Klestrup Röijezon wrote: > >> Hi, >> >> A JRE should be enough for running it, but you need sbt and a JDK for >> building. I've got a derivation for a Play website at >> https://github.com/BlocklandGlass/ParseTS-Playground/blob/master/parsets-playground.nix, >> with the NixOS/NixOps setup at >> https://github.com/BlocklandGlass/ParseTS-Playground/tree/master/deployment >> . >> >> The gist of it is to run "sbt stage" in the build phase, and to then take >> "target/universal/stage" as your build output. However, you'll also need to >> wrap the launcher script to add your JRE and to add gawk (which the >> launcher script requires). Finally, on any modern system (such as NixOS) >> you'll also want to disable Play's PID file management, since systemd takes >> care of that anyway. I didn't in that script, but you'll probably also want >> to add a testing phase as part of the build. >> >> The big drawback with this approach is that SBT downloads all >> dependencies from the internet on demand, which won't work on a Nix setup >> with proper isolation (ideally, builds should only have network access if >> they deterministically produce a given hash). >> >> I've been toying with the idea of writing a sbt2nix SBT plugin that >> generates Nix definitions to build a local maven mirror for the >> dependencies, but I haven't got around to that (yet). >> >> // Teo >> >> On 5 July 2016 at 21:52, 4levels <4lev...@gmail.com> wrote: >> >>> Hi Nix-devs, >>> >>> This is a plain request for assistance / best practices for using Nixos >>> with Java / Scala / Play. Akka with EventSourcing are also a topic of >>> interest. >>> >>> I'm currently trying to get a Scala Play app up and running on my nixOps >>> deployed machines. As I'm very unfamiliar with running Java based apps, >>> I'd like to know if someone has experience on the common pitfalls and tips >>> on keeping the servers healthy (I just caused my laptop's 8 cores to go >>> 100% without being able to stop the server started by the activator call). >>> >>> I've seen some related packages in nixpkgs and have many questions like >>> eg. do I need sbt (which seems to provide typesafe - activator) and a jdk >>> on the production servers or are is a jre sufficient? How do I deploy and >>> run a Java app developed locally? >>> And how do I set-up a local nixos vm for Java development? >>> >>> I'm still investigating and learning a lot myself, so nix-related >>> knowledge is my main concern here (as I need to figure out the rest myself >>> anyway ;-) >>> >>> I'll be happy to share my findings and configuration / setup.. >>> >>> >>> Kind regards, >>> >>> Erik >>> >>> ___ >>> nix-dev mailing list >>> nix-dev@lists.science.uu.nl >>> http://lists.science.uu.nl/mailman/listinfo/nix-dev >>> >>> >> ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Tips on deploying a Scala Play application
HI Erik, That's pretty much entirely wrong. :P ParseTS is just a linter script for the game scripting language TorqueScript. ParseTS-Playground was a pastebin that would run the submitted code through the linter. For example, see https://parsets-playground.nullable.se/snippets/13. The datastore used was PostgreSQL. Anyway, apart from the ParseTS stuff, at least those scripts should be pretty much straightforward to copy to any Play application, though for the config stuff to work you'll need to add the line 'include "local.conf"' to your conf/application.conf. Any chance you could post your current setup and the errors you get? // Teo On 6 July 2016 at 04:31, 4levels <4lev...@gmail.com> wrote: > Hi Teo, > > I've come quite far in setting up things, but I keep running into building > errors. > It has everything to do with me removing all references to parsets and > postgres and renaming things here and there, trying to merge them with the > current deploy setup. > > Do I understand correctly that parsets is a library to store data, using > postgres in the background? I'd like to start using Event Sourcing with > Scala / Akka so I don't need a datastore like parsets, correct? I'm very > unsure about this as I literally started today with learning Scala / Play. > I got my toes wet with Java before but that's really it. > > Something else I found interesting as I'm quite an Nginx fan and have > nginx running with proxies already: Nginx has capabilities to deal with > Java in different ways, as proxy or tied with eg Clojure for even faster > results.. > > The journey continues ;-) > > > Kind regards, > > Erik > > On Tue, Jul 5, 2016 at 10:23 PM 4levels <4lev...@gmail.com> wrote: > >> Hi Teo, >> >> Thank you for your explanation and quick qualitative response! >> >> I'll be looking at your code asap and report back with my experiences ;-) >> >> Kind regards, >> >> Erik >> >> On Tue, Jul 5, 2016, 22:08 Teo Klestrup Röijezon wrote: >> >>> Hi, >>> >>> A JRE should be enough for running it, but you need sbt and a JDK for >>> building. I've got a derivation for a Play website at >>> https://github.com/BlocklandGlass/ParseTS-Playground/blob/master/parsets-playground.nix, >>> with the NixOS/NixOps setup at >>> https://github.com/BlocklandGlass/ParseTS-Playground/tree/master/deployment >>> . >>> >>> The gist of it is to run "sbt stage" in the build phase, and to then >>> take "target/universal/stage" as your build output. However, you'll also >>> need to wrap the launcher script to add your JRE and to add gawk (which the >>> launcher script requires). Finally, on any modern system (such as NixOS) >>> you'll also want to disable Play's PID file management, since systemd takes >>> care of that anyway. I didn't in that script, but you'll probably also want >>> to add a testing phase as part of the build. >>> >>> The big drawback with this approach is that SBT downloads all >>> dependencies from the internet on demand, which won't work on a Nix setup >>> with proper isolation (ideally, builds should only have network access if >>> they deterministically produce a given hash). >>> >>> I've been toying with the idea of writing a sbt2nix SBT plugin that >>> generates Nix definitions to build a local maven mirror for the >>> dependencies, but I haven't got around to that (yet). >>> >>> // Teo >>> >>> On 5 July 2016 at 21:52, 4levels <4lev...@gmail.com> wrote: >>> Hi Nix-devs, This is a plain request for assistance / best practices for using Nixos with Java / Scala / Play. Akka with EventSourcing are also a topic of interest. I'm currently trying to get a Scala Play app up and running on my nixOps deployed machines. As I'm very unfamiliar with running Java based apps, I'd like to know if someone has experience on the common pitfalls and tips on keeping the servers healthy (I just caused my laptop's 8 cores to go 100% without being able to stop the server started by the activator call). I've seen some related packages in nixpkgs and have many questions like eg. do I need sbt (which seems to provide typesafe - activator) and a jdk on the production servers or are is a jre sufficient? How do I deploy and run a Java app developed locally? And how do I set-up a local nixos vm for Java development? I'm still investigating and learning a lot myself, so nix-related knowledge is my main concern here (as I need to figure out the rest myself anyway ;-) I'll be happy to share my findings and configuration / setup.. Kind regards, Erik ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev >>> ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Tips on deploying a Scala Play application
Hi Teo,
I knew I was getting off course with my conclusions, thanks for clarifying
this!
I'll try to give you an overview, I don't mind adding you to our private
Bitbucket repo if you'd like to see all files and folders.
I still don't know where I should add the statements to have the play
project deployed. All I have for now is the project's directory in a
subfolder, src/play.mancloud.eu
I still have many questions regarding nixo(p)s internals:
- how does the sequence of the nixops modify files matter?
- when to use with *import ;* or *{ stdenv, lib, config, pkgs, ...
}:* and what are the differences
- ..
I'll try to give you an explanation of how the deploy scripts are composed
below. There's still a lot of room for improvements and regrouping of
statements as I'm still a nix beginner..
*nixops info* output
Nix expressions: vultr.nix defaults-local.nix defaults.nix
servers-local.nix keys-vm01.nix platform-local.nix
*vultr.nix* contains Vultr specifics + the collectd setup
{
defaults = {
deployment = {
targetEnv = "none";
};
fileSystems."/" =
{
device = "/dev/vda1";
options = [ "noatime" "nodiratime" "discard" ];
};
swapDevices = [
{
device = "/dev/vda2";
}
];
boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd"
"virtio_pci" "virtio_blk" ];
boot.loader.grub.enable = true;
boot.loader.grub.version = 2;
boot.loader.grub.device = "/dev/vda";
services.collectd.extraConfig = ''
Device "/dev/vda1"
MountPoint "/"
Interface "enp0s3"
IgnoreSelected false
'';
};
}
*defaults-local.nix* contains some tweaks for my local vm as well as the
copied call to parsets (which I renamed to mancloud-play)
with import ;
{
defaults =
{ stdenv, lib, config, pkgs, ... }:
let
wrapper = pkgs.callPackage ./mancloud-play-wrapper.nix {
mancloud-play = import ./mancloud-play-package.nix;
port = 9000;
};
in
{
programs.bash.promptInit = ''
# Provide a nice prompt if the terminal supports it.
if [ "$TERM" != "dumb" -o -n "$INSIDE_EMACS" ]; then
PROMPT_COLOR="1;34m"
let $UID && PROMPT_COLOR="1;34m"
PS1="\n\[\033[$PROMPT_COLOR\][\u@\h:\w]\\$\[\033[0m\] "
if test "$TERM" = "xterm"; then
PS1="\[\033]3;\h:\u:\w\007\]$PS1"
fi
fi
'';
systemd.services = {
inherit (wrapper.systemd.services) mancloud-play;
};
};
}
*defaults.nix* contains quite some services (key deployments, hostfile
generation based on active projects, the mancloud (PHP) package, PHP-FPM
tweaks and an alert service to monitor our queues, users, nginx, mysql,
collectd, fail2ban, .. I've removed quite some config as this file is
currently 540 lines long
with import ;
{
defaults =
{ config, pkgs, lib, nodes, ... }:
let
serverKeys = keys:
lib.genAttrs keys (n:
{
text = lib.removeSuffix "\n" (builtins.readFile (./keys +
"/${builtins.replaceStrings ["@"] ["-"] n}") );
group = "keys";
permissions = "0640";
}
)
;
in
{
deployment.keys = serverKeys [
"mancloud.amazon.iam.key_id"
"mancloud.amazon.iam.access_key"
"mancloud.amazon.iam.passphrase"
"mancloud._test.password"
"phpmyadmin.password"
"phpmyadmin.secret"
];
environment.systemPackages = with pkgs; [
wget
unzip
gitMinimal
tmux
mariadb
php
duplicity
nodejs
redis
phpPackages.composer
phpPackages.redis
letsencrypt
];
};
}
*servers-local.nix* contains the specific machine declarations (locally 1
machine vm01) like ssl config, nfs mounts, libvirt related network config
and some overrides from the defaults (stiil some reorganisation needed).
It imports the extra service configuration (here diem-service.nix)
with import ;
let
sslCert = ./src/ssl-cert.pem;
sslKey = ./src/ssl-key.pem;
sslCA = ./src/ssl-ca.pem;
sslDHParam = ./src/ssl-dhparam.pem;
nginxSslConfig = ''
${builtins.readFile ./src/nginx-ssl.conf}
ssl_certificate ${sslCert};
ssl_certificate_key ${sslKey};
ssl_trusted_certificate ${sslCA};
ssl_dhparam ${sslDHParam};
'';
in
{
vm01 =
{ config, pkgs, nodes, ... }:
{
imports = [ ./diem-service.nix ];
deployment = {
targetHost = "192.168.121.50";
};
environment.systemPackages = with pkgs; [
strace
gitAndTools.git-crypt
];
services.mysql.enable = true;
networking.hostName = "vm01"; # Define your hostname.
networking.enableIPv6 = false;
networking.extraHosts = "192.168.121.1 d01 d01.local";
networking.interfaces = {
enp0s3 = {
ip4 = [ { address = "192.168.121.50"; prefixLength = 24; } ];
};
enp0s9 = {
ip4 = [ { address = "192.168.0.98"; prefixLe
Re: [Nix-dev] Tips on deploying a Scala Play application
Hi Teo,
My previous email is too long and being considered for moderation by the
nix-dev mailing list.
I forgot to also include the files based on your code. This is where the
build fails:
./mancloud-play-wrapper.nix:37:17 - the script line of the systemd.service
declaration
*mancloud-play-wrapper.nix* is the wrapper script from your deployment
subfolder
{ stdenv, writeText, writeScript, makeWrapper
, bash
, mancloud-play
, port
, lib
, useRecaptcha ? false
, recaptchaSiteKey ? null
, recaptchaSecretKey ? null
, reverseProxyIp ? null
}:
assert useRecaptcha -> (recaptchaSiteKey != null && recaptchaSecretKey != null);
let
configFile = writeText "mancloud-play.conf"
''
include "application.conf"
${lib.optionalString (reverseProxyIp != null)
"play.http.forwarded.trustedProxies += ['${reverseProxyIp}']"}
${lib.optionalString useRecaptcha
''
captcha {
provider = "recaptcha"
recaptcha {
siteKey = ${recaptchaSiteKey}
secretKey = ${recaptchaSecretKey}
}
}
''
}
'';
in {
systemd.services = {
mancloud-play = {
wantedBy = [ "multi-user.target" ];
requires = [ ];
after = [ "network.target" ];
script = "exec ${mancloud-play}/bin/mancloud-play
-Dconfig.file=${configFile} -Dhttp.port=${toString port}";
};
};
}
*mancloud-play-package.nix* contains your packaging code
with import {};
{
pidFile ? "/dev/null"
, devMode ? false
, extraConfig ? ""
}:
assert (builtins.isBool devMode);
let
localDevConfig = ./conf/local.conf;
generatedConfig = writeText "mancloud-play.conf" ''
${builtins.optionalString devMode "include \"local.dev.conf\""}
${extraConfig}
'';
in
stdenv.mkDerivation rec {
name= "mancloud-play";
src = ./src/play.mancloud.eu;
buildInputs = [ jre jdk sbt makeWrapper ];
configurePhase =
''
${builtins.optionalString devMode "cp ${localDevConfig}
conf/local.dev.conf"}
ln -s ${generatedConfig} conf/local.conf
sbt playUpdateSecret
'';
buildPhase = "sbt stage";
installPhase =
''
cp -r target/universal/stage $out
wrapProgram $out/bin/mancloud-play \
--set JAVA_HOME ${jre} \
--suffix PATH : "${gawk}/bin" \
--add-flags \
-Dpidfile.path=${pidFile}
mkdir -p $out/etc
cat >$out/etc/process_config < wrote:
> Hi Teo,
>
> I knew I was getting off course with my conclusions, thanks for clarifying
> this!
>
> I'll try to give you an overview, I don't mind adding you to our private
> Bitbucket repo if you'd like to see all files and folders.
> I still don't know where I should add the statements to have the play
> project deployed. All I have for now is the project's directory in a
> subfolder, src/play.mancloud.eu
>
> I still have many questions regarding nixo(p)s internals:
> - how does the sequence of the nixops modify files matter?
> - when to use with *import ;* or *{ stdenv, lib, config, pkgs,
> ... }:* and what are the differences
> - ..
>
> I'll try to give you an explanation of how the deploy scripts are composed
> below. There's still a lot of room for improvements and regrouping of
> statements as I'm still a nix beginner..
>
>
> *nixops info* output
> Nix expressions: vultr.nix defaults-local.nix defaults.nix
> servers-local.nix keys-vm01.nix platform-local.nix
>
> *vultr.nix* contains Vultr specifics + the collectd setup
>
> {
> defaults = {
> deployment = {
> targetEnv = "none";
> };
> fileSystems."/" =
> {
> device = "/dev/vda1";
> options = [ "noatime" "nodiratime" "discard" ];
> };
> swapDevices = [
> {
> device = "/dev/vda2";
> }
> ];
> boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" "virtio_pci"
> "virtio_blk" ];
> boot.loader.grub.enable = true;
> boot.loader.grub.version = 2;
> boot.loader.grub.device = "/dev/vda";
> services.collectd.extraConfig = ''
>
> Device "/dev/vda1"
> MountPoint "/"
>
>
> Interface "enp0s3"
> IgnoreSelected false
>
> '';
> };
> }
>
> *defaults-local.nix* contains some tweaks for my local vm as well as the
> copied call to parsets (which I renamed to mancloud-play)
>
> with import ;
> {
> defaults =
> { stdenv, lib, config, pkgs, ... }:
> let
> wrapper = pkgs.callPackage ./mancloud-play-wrapper.nix {
> mancloud-play = import ./mancloud-play-package.nix;
> port = 9000;
> };
> in
> {
> programs.bash.promptInit = ''
> # Provide a nice prompt if the terminal supports it.
> if [ "$TERM" != "dumb" -o -n "$INSIDE_EMACS" ]; then
> PROMPT_COLOR="1;34m"
> let $UID && PROMPT_COLOR="1;34m"
> PS1="\n\[\033[$PROMPT_COLOR\][\u@\h:\w]\\$\[\033[0m\] "
> if test "$TERM" = "xterm"; then
> PS1="\[\033]3;\h:\u:\w\007\]$PS1"
> fi
> fi
> '';
> systemd.services = {
> inherit (wrapper
Re: [Nix-dev] Tips on deploying a Scala Play application
Hey Erik,
> how does the sequence of the nixops modify files matter?
The files listed are merged when building the actual NixOS configurations.
> when to use with *import ;* or *{ stdenv, lib, config, pkgs, ...
}:* and what are the differences
I think there are actually three different contexts to keep in mind here:
packages definitions, NixOS modules, and standalone nix definitions. Quick
disclaimer: I am by no means an expert either.
For regular packages, you should take the dependencies as arguments. That
makes it easy to override them with, say, patched versions. Also, the
channel might not always be named nixpkgs, and it might not even be in
NIX_PATH at all.
For NixOS modules (including when part of NixOps network definitions), the
same concerns apply, and Nixpkgs as a whole should instead be injected
through the `pkgs` argument. I'm not sure, but I don't believe the `config`
is accessible at all using `import`.
Finally, for standalone nix definitions (mostly useful for the
`default.nix` used for development builds), you'll need to use `import`.
Regarding the error, there's not really much that stands out to me right
now, except for that you shouldn't use `exec` in systemd services. That
shouldn't cause a build error, though.
// Teo
On 6 July 2016 at 17:10, 4levels <4lev...@gmail.com> wrote:
> Hi Teo,
>
> I knew I was getting off course with my conclusions, thanks for clarifying
> this!
>
> I'll try to give you an overview, I don't mind adding you to our private
> Bitbucket repo if you'd like to see all files and folders.
> I still don't know where I should add the statements to have the play
> project deployed. All I have for now is the project's directory in a
> subfolder, src/play.mancloud.eu
>
> I still have many questions regarding nixo(p)s internals:
> - how does the sequence of the nixops modify files matter?
> - when to use with *import ;* or *{ stdenv, lib, config, pkgs,
> ... }:* and what are the differences
> - ..
>
> I'll try to give you an explanation of how the deploy scripts are composed
> below. There's still a lot of room for improvements and regrouping of
> statements as I'm still a nix beginner..
>
>
> *nixops info* output
> Nix expressions: vultr.nix defaults-local.nix defaults.nix
> servers-local.nix keys-vm01.nix platform-local.nix
>
> *vultr.nix* contains Vultr specifics + the collectd setup
>
> {
> defaults = {
> deployment = {
> targetEnv = "none";
> };
> fileSystems."/" =
> {
> device = "/dev/vda1";
> options = [ "noatime" "nodiratime" "discard" ];
> };
> swapDevices = [
> {
> device = "/dev/vda2";
> }
> ];
> boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" "virtio_pci"
> "virtio_blk" ];
> boot.loader.grub.enable = true;
> boot.loader.grub.version = 2;
> boot.loader.grub.device = "/dev/vda";
> services.collectd.extraConfig = ''
>
> Device "/dev/vda1"
> MountPoint "/"
>
>
> Interface "enp0s3"
> IgnoreSelected false
>
> '';
> };
> }
>
> *defaults-local.nix* contains some tweaks for my local vm as well as the
> copied call to parsets (which I renamed to mancloud-play)
>
> with import ;
> {
> defaults =
> { stdenv, lib, config, pkgs, ... }:
> let
> wrapper = pkgs.callPackage ./mancloud-play-wrapper.nix {
> mancloud-play = import ./mancloud-play-package.nix;
> port = 9000;
> };
> in
> {
> programs.bash.promptInit = ''
> # Provide a nice prompt if the terminal supports it.
> if [ "$TERM" != "dumb" -o -n "$INSIDE_EMACS" ]; then
> PROMPT_COLOR="1;34m"
> let $UID && PROMPT_COLOR="1;34m"
> PS1="\n\[\033[$PROMPT_COLOR\][\u@\h:\w]\\$\[\033[0m\] "
> if test "$TERM" = "xterm"; then
> PS1="\[\033]3;\h:\u:\w\007\]$PS1"
> fi
> fi
> '';
> systemd.services = {
> inherit (wrapper.systemd.services) mancloud-play;
> };
> };
> }
>
> *defaults.nix* contains quite some services (key deployments, hostfile
> generation based on active projects, the mancloud (PHP) package, PHP-FPM
> tweaks and an alert service to monitor our queues, users, nginx, mysql,
> collectd, fail2ban, .. I've removed quite some config as this file is
> currently 540 lines long
>
> with import ;
> {
> defaults =
> { config, pkgs, lib, nodes, ... }:
> let
> serverKeys = keys:
> lib.genAttrs keys (n:
> {
> text = lib.removeSuffix "\n" (builtins.readFile (./keys +
> "/${builtins.replaceStrings ["@"] ["-"] n}") );
> group = "keys";
> permissions = "0640";
> }
> )
> ;
>
>
> in
> {
> deployment.keys = serverKeys [
> "mancloud.amazon.iam.key_id"
> "mancloud.amazon.iam.access_key"
> "mancloud.amazon.iam.passphrase"
> "mancloud._test.password"
> "phpmyadmin.password"
> "phpmyadmin.secret"
> ];
> environment.systemP
Re: [Nix-dev] Tips on deploying a Scala Play application
Hi Teo,
thank you for the brief introduction into the different contexts, I was not
aware of this at all and was using trial-and-error to get things running.
I still need to learn a lot here, that's for sure!
Regarding parseTS: do I need to setup something similar to your parsets.nix
(adjusted for my needs) or can I skip this completely? In other words, in
your github repo here
https://github.com/BlocklandGlass/ParseTS-Playground which
files do I need to adjust / implement? Do I understand correctly that I'll
have to look at these 3 files?
- defaults.nix
- parsets.nix
- parsets-playground.nix
The file parsets-plaground-wrapper.nix in your deployment folder is
sufficient for the nixops part of this story, right?
Too bad you couldn't spot the issue, did you receive both my emails (since
one was being held for moderation)?
Regarding the exec in the systemd definition: this is what I copied from
your code here:
https://github.com/BlocklandGlass/ParseTS-Playground/blob/master/deployment/parsets-playground-wrapper.nix
I'm still trying to get in touch with some other people who might be able
to shed some light on how to deploy a Play application correctly. I'm glad
to see that you pulled it off in your demo on nullable.se ;-)
Thanks again for your support!
Erik
On Wed, Jul 6, 2016 at 10:00 PM Teo Klestrup Röijezon
wrote:
> Hey Erik,
>
> > how does the sequence of the nixops modify files matter?
>
> The files listed are merged when building the actual NixOS configurations.
>
> > when to use with *import ;* or *{ stdenv, lib, config, pkgs,
> ... }:* and what are the differences
>
> I think there are actually three different contexts to keep in mind here:
> packages definitions, NixOS modules, and standalone nix definitions. Quick
> disclaimer: I am by no means an expert either.
>
> For regular packages, you should take the dependencies as arguments. That
> makes it easy to override them with, say, patched versions. Also, the
> channel might not always be named nixpkgs, and it might not even be in
> NIX_PATH at all.
>
> For NixOS modules (including when part of NixOps network definitions), the
> same concerns apply, and Nixpkgs as a whole should instead be injected
> through the `pkgs` argument. I'm not sure, but I don't believe the `config`
> is accessible at all using `import`.
>
> Finally, for standalone nix definitions (mostly useful for the
> `default.nix` used for development builds), you'll need to use `import`.
>
> Regarding the error, there's not really much that stands out to me right
> now, except for that you shouldn't use `exec` in systemd services. That
> shouldn't cause a build error, though.
>
> // Teo
>
> On 6 July 2016 at 17:10, 4levels <4lev...@gmail.com> wrote:
>
>> Hi Teo,
>>
>> I knew I was getting off course with my conclusions, thanks for
>> clarifying this!
>>
>> I'll try to give you an overview, I don't mind adding you to our private
>> Bitbucket repo if you'd like to see all files and folders.
>> I still don't know where I should add the statements to have the play
>> project deployed. All I have for now is the project's directory in a
>> subfolder, src/play.mancloud.eu
>>
>> I still have many questions regarding nixo(p)s internals:
>> - how does the sequence of the nixops modify files matter?
>> - when to use with *import ;* or *{ stdenv, lib, config, pkgs,
>> ... }:* and what are the differences
>> - ..
>>
>> I'll try to give you an explanation of how the deploy scripts are
>> composed below. There's still a lot of room for improvements and
>> regrouping of statements as I'm still a nix beginner..
>>
>>
>> *nixops info* output
>> Nix expressions: vultr.nix defaults-local.nix defaults.nix
>> servers-local.nix keys-vm01.nix platform-local.nix
>>
>> *vultr.nix* contains Vultr specifics + the collectd setup
>>
>> {
>> defaults = {
>> deployment = {
>> targetEnv = "none";
>> };
>> fileSystems."/" =
>> {
>> device = "/dev/vda1";
>> options = [ "noatime" "nodiratime" "discard" ];
>> };
>> swapDevices = [
>> {
>> device = "/dev/vda2";
>> }
>> ];
>> boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd"
>> "virtio_pci" "virtio_blk" ];
>> boot.loader.grub.enable = true;
>> boot.loader.grub.version = 2;
>> boot.loader.grub.device = "/dev/vda";
>> services.collectd.extraConfig = ''
>>
>> Device "/dev/vda1"
>> MountPoint "/"
>>
>>
>> Interface "enp0s3"
>> IgnoreSelected false
>>
>> '';
>> };
>> }
>>
>> *defaults-local.nix* contains some tweaks for my local vm as well as the
>> copied call to parsets (which I renamed to mancloud-play)
>>
>> with import ;
>> {
>> defaults =
>> { stdenv, lib, config, pkgs, ... }:
>> let
>> wrapper = pkgs.callPackage ./mancloud-play-wrapper.nix {
>> mancloud-play = import ./mancloud-play-package.nix;
>> port = 9000;
>> };
>> in
>> {
>> programs.bash.promptInit = ''
>> # Provide a nice prompt if the te
Re: [Nix-dev] Tips on deploying a Scala Play application
Hi Erik,
Strictly speaking then parsets-playground.nix is the only one that's
required here. parsets.nix is for the ParseTS binary itself (a
run-of-the-mill Haskell project) and default.nix is only there to get
nix-build and nix-shell working. Didn't realize that the exec part was
there in mine too, sorry about that. Anyway, what was the error message it
gives you?
Perhaps it would be better to take this over IRC instead, to avoid spamming
the mailing list with too much back-and-forth.
// Teo
On 7 July 2016 at 09:44, 4levels <4lev...@gmail.com> wrote:
> Hi Teo,
>
> thank you for the brief introduction into the different contexts, I was
> not aware of this at all and was using trial-and-error to get things
> running.
> I still need to learn a lot here, that's for sure!
>
> Regarding parseTS: do I need to setup something similar to your
> parsets.nix (adjusted for my needs) or can I skip this completely? In
> other words, in your github repo here
> https://github.com/BlocklandGlass/ParseTS-Playground which files do I
> need to adjust / implement? Do I understand correctly that I'll have to
> look at these 3 files?
> - defaults.nix
> - parsets.nix
> - parsets-playground.nix
>
> The file parsets-plaground-wrapper.nix in your deployment folder is
> sufficient for the nixops part of this story, right?
>
> Too bad you couldn't spot the issue, did you receive both my emails (since
> one was being held for moderation)?
>
> Regarding the exec in the systemd definition: this is what I copied from
> your code here:
> https://github.com/BlocklandGlass/ParseTS-Playground/blob/master/deployment/parsets-playground-wrapper.nix
>
> I'm still trying to get in touch with some other people who might be able
> to shed some light on how to deploy a Play application correctly. I'm glad
> to see that you pulled it off in your demo on nullable.se ;-)
>
>
> Thanks again for your support!
>
> Erik
>
> On Wed, Jul 6, 2016 at 10:00 PM Teo Klestrup Röijezon
> wrote:
>
>> Hey Erik,
>>
>> > how does the sequence of the nixops modify files matter?
>>
>> The files listed are merged when building the actual NixOS configurations.
>>
>> > when to use with *import ;* or *{ stdenv, lib, config, pkgs,
>> ... }:* and what are the differences
>>
>> I think there are actually three different contexts to keep in mind here:
>> packages definitions, NixOS modules, and standalone nix definitions. Quick
>> disclaimer: I am by no means an expert either.
>>
>> For regular packages, you should take the dependencies as arguments. That
>> makes it easy to override them with, say, patched versions. Also, the
>> channel might not always be named nixpkgs, and it might not even be in
>> NIX_PATH at all.
>>
>> For NixOS modules (including when part of NixOps network definitions),
>> the same concerns apply, and Nixpkgs as a whole should instead be injected
>> through the `pkgs` argument. I'm not sure, but I don't believe the `config`
>> is accessible at all using `import`.
>>
>> Finally, for standalone nix definitions (mostly useful for the
>> `default.nix` used for development builds), you'll need to use `import`.
>>
>> Regarding the error, there's not really much that stands out to me right
>> now, except for that you shouldn't use `exec` in systemd services. That
>> shouldn't cause a build error, though.
>>
>> // Teo
>>
>> On 6 July 2016 at 17:10, 4levels <4lev...@gmail.com> wrote:
>>
>>> Hi Teo,
>>>
>>> I knew I was getting off course with my conclusions, thanks for
>>> clarifying this!
>>>
>>> I'll try to give you an overview, I don't mind adding you to our private
>>> Bitbucket repo if you'd like to see all files and folders.
>>> I still don't know where I should add the statements to have the play
>>> project deployed. All I have for now is the project's directory in a
>>> subfolder, src/play.mancloud.eu
>>>
>>> I still have many questions regarding nixo(p)s internals:
>>> - how does the sequence of the nixops modify files matter?
>>> - when to use with *import ;* or *{ stdenv, lib, config, pkgs,
>>> ... }:* and what are the differences
>>> - ..
>>>
>>> I'll try to give you an explanation of how the deploy scripts are
>>> composed below. There's still a lot of room for improvements and
>>> regrouping of statements as I'm still a nix beginner..
>>>
>>>
>>> *nixops info* output
>>> Nix expressions: vultr.nix defaults-local.nix defaults.nix
>>> servers-local.nix keys-vm01.nix platform-local.nix
>>>
>>> *vultr.nix* contains Vultr specifics + the collectd setup
>>>
>>> {
>>> defaults = {
>>> deployment = {
>>> targetEnv = "none";
>>> };
>>> fileSystems."/" =
>>> {
>>> device = "/dev/vda1";
>>> options = [ "noatime" "nodiratime" "discard" ];
>>> };
>>> swapDevices = [
>>> {
>>> device = "/dev/vda2";
>>> }
>>> ];
>>> boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd"
>>> "virtio_pci" "virtio_blk" ];
>>> boot.loader.grub.enable = true;
>>> boot.loader.grub.version =
