NixOps deploys this blog on AWS

2020-05-30

NixOS allows us to declaratively configure a Linux machine.
NixOps allows us to declaratively provision and deploy a NixOS instance to the cloud.

What does all of this mean?

That with a single plain text file, I can have this blog running on AWS.

In this post, I'm going to show and explain the steps for doing this.

Setup
- Nix File
Provisioning
Deployment
- NixOS on EC2
- Nginx
- Blog
Destroying

Setup ¶

For replicating the steps, you should have:

An AWS account
The Nix package manager
An ~/.aws/credentials file. This can be created with the AWS CLI.
NixOps 1.7
- Install it:
  nix-env -i nixops
- Check its version:
  nixops --version

Nix File ¶

We're going to start off by creating the plain text file, we'll call it deploy.nix. This file is a Nix Language expression, it will contain all the specification needed for provisioning and deploying the NixOS instance.

Initially, we'll just include the AWS region and profile in the file.


# deploy.nix
let
  region = "us-east-2";
  accessKeyId = "default"; # "default" is the profile in ~/.aws/credentials
in {
  network.description = "stevechavez.xyz"; # Optional description
}

Now we'll this register this file to NixOps.


$ nixops create -d xyz deploy.nix # xyz is a shortcut for our deployment

created deployment ‘36d5282e-94b8-11ea-a0f2-047d7bf5e9c4’
36d5282e-94b8-11ea-a0f2-047d7bf5e9c4

We do this because NixOps needs to track the state of the deployment. The state is stored in a SQLite database, which lives in ~/.nixops/deployments.nixops¹.

Provisioning ¶

Let's now move on to AWS stuff. To get our NixOS running, we need to provision the resources that the machine will use.

EC2 Key Pair ¶

To be able to access our NixOS instance on AWS, we'll need an EC2 Key pair.


# deploy.nix
let
  region = "us-east-2";
  accessKeyId = "default";
in {
  network.description = "stevechavez.xyz";

+ resources = {
+   ec2KeyPairs.xyzKeyPair = { inherit region accessKeyId; };
+ };

}

Here we are adding an EC2 key pair called xyzKeyPair(can be any name). The resources set and the attributes it can take are defined by NixOps. This is documented on the NixOps User's Guide.

Now we'll start pushing our config to AWS. We can do this with the deploy command.


$ nixops deploy -d xyz

xyzKeyPair> uploading EC2 key pair ‘charon-36d5282e-94b8-11ea-a0f2-047d7bf5e9c4-xyzKeyPair’...
...
xyz> deployment finished successfully

So, NixOps did his job successfully. But, where did the key pair go?

As mentioned before, NixOps uses a SQLite db. We can check this db with:


$ sqlite3 ~/.nixops/deployments.nixops .dump

# ...
# Some details omitted
INSERT INTO ResourceAttrs VALUES(50,'publicKey','ssh-rsa xxxx...xxx');
INSERT INTO ResourceAttrs VALUES(50,'privateKey',replace('-----BEGIN OPENSSH PRIVATE KEY-----\nxxx...xxx\n-----END OPENSSH PRIVATE KEY-----\n','\n',char(10)));
# ...

Here we can see the public and private key. If you used AWS before, the private key would usualy be stored in a pem file.

Security Group ¶

To reach our NixOS instance, we'll need to configure its network.

AWS offers many options for this, but here we'll just use the default VPC(can be thought as a VLAN) and add a Security Group(can be thought as a firewall).


# deploy.nix
let
  region = "us-east-2";
  accessKeyId = "default";
in {
  network.description = "stevechavez.xyz";
  resources = {
    ec2KeyPairs.xyzKeyPair = { inherit region accessKeyId; };

+   ec2SecurityGroups.xyzGroup = {
+     inherit region accessKeyId;
+     name        = "xyz-group";
+     description = "stevechavez.xyz security group";
+     rules = [
+       { fromPort = 22;  toPort = 22;  sourceIp = "0.0.0.0/0"; }
+       { fromPort = 80;  toPort = 80;  sourceIp = "0.0.0.0/0"; }
+     ];
+   };
+ };

}

Here we opened the http port(80) and ssh port(22) to the public. In the ssh case, we could be extra safe by only allowing to connect to it from our IP(using sourceIp = "yo.u.r.ip/32"). But we'll keep it like this for this post. We're safe enough because we can only login with our private key.

Let's now create the Security Group in AWS:


$ nixops deploy -d xyz

xyzGroup..> creating EC2 security group ‘xyz-group’...
xyzGroup..> adding new rules to EC2 security group ‘xyz-group’...
building all machine configurations...
xyz> closures copied successfully
xyz> deployment finished successfully

This change was also stored in the SQLite db. This time, instead of dumping the db, we can look at some of its contents with the nixops info command.


$ nixops info -d xyz

Network name: xyz
Network UUID: 36d5282e-94b8-11ea-a0f2-047d7bf5e9c4
Network description: xyz
Nix expressions: /home/steve-chavez/Projects/stevechavez.xyz/deploy.nix

+------------+--------+--------------------------------+--------------------------------------------------------+------------+
| Name       | Status | Type                           | Resource Id                                            | IP address |
+------------+--------+--------------------------------+--------------------------------------------------------+------------+
| xyzKeyPair |   Up   | ec2-keypair [us-east-2]        | charon-36d5282e-94b8-11ea-a0f2-047d7bf5e9c4-xyzKeyPair |            |
| xyzGroup   |   Up   | ec2-security-group [us-east-2] | xyz-group                                              |            |
+------------+--------+--------------------------------+--------------------------------------------------------+------------+

Here we can see the resources we created until now.

Elastic IP ¶

To finish provisioning, we'll create an Elastic IP(can be thought as a public IP).


# deploy.nix
let
  region = "us-east-2";
  accessKeyId = "default";
in {
  network.description = "xyz";

  resources = {
    ec2KeyPairs.xyzKeyPair = { inherit region accessKeyId; };

    ec2SecurityGroups.xyzGroup = {
      inherit region accessKeyId;
      name        = "xyz-group";
      description = "stevechavez.xyz security group";
      rules = [
        { fromPort = 22;  toPort = 22;  sourceIp = "0.0.0.0/0"; }
        { fromPort = 80;  toPort = 80;  sourceIp = "0.0.0.0/0"; }
        { fromPort = 443; toPort = 443; sourceIp = "0.0.0.0/0"; }
      ];

+   elasticIPs.xyzIP = { inherit region accessKeyId; };

    };
  };
}

And we'll run nixops deploy again to create it:


$ nixops deploy -d xyz

xyzIP.....> creating elastic IP address (region ‘us-east-2’ - domain ‘standard’)...
xyzIP.....> IP address is 3.23.156.242
building all machine configurations...
xyz> closures copied successfully

We need the Elastic IP for associating it with the domain name. This can be done by adding the IP as an A record to the domain registrar's DNS. This involves a manual setup, so we'll not cover it here.

Deployment ¶

Now we're going to deploy our NixOS instance and start configuring it.

NixOS on EC2 ¶

Let's create the EC2 instance(a virtual server), deploy NixOS on it, and assign it all of our previously created resources.


# deploy.nix
let
  region = "us-east-2";
  accessKeyId = "default";
in {
  network.description = "stevechavez.xyz";

  resources = {
    ...
  };

+ main = { resources, ... }: {
+   deployment = {
+     targetEnv = "ec2";
+     ec2 = {
+       inherit region accessKeyId;
+       instanceType             = "t2.nano";
+       keyPair                  = resources.ec2KeyPairs.xyzKeyPair;
+       elasticIPv4              = resources.elasticIPs.xyzIP;
+       securityGroups           = [ resources.ec2SecurityGroups.xyzGroup ];
+     };
+   };
+ };

}


$ nixops deploy -d xyz

main.....> creating EC2 instance (AMI ‘ami-093efd3a57a1e03a8’, type ‘t2.nano’, region ‘us-east-2’)...
main.....> waiting for machine to be in running state... [pending] [pending] [pending] [pending] [running]
main.....> associating IP address ‘3.23.156.242’...
main.....> waiting for address to be associated with this machine... [3.23.156.242]
main.....> waiting for IP address... [running] 3.23.156.242 / 172.31.43.244
main.....> waiting for SSH.....................................
main.....> replacing temporary host key...
building all machine configurations...
...

main.....> starting the following units: apply-ec2-data.service, audit.service... # more services here
main.....> activation finished successfully

Here NixOps deployed the NixOS AMI(ami-093efd3a57a1e03a8) to a t2.nano instance. This image includes a basic default config.

Let's ssh into the machine and see a sample of of its running services:


$ nixops ssh -d xyz main

[root@main:~]#
[root@main:~]# systemctl list-units --state=running

UNIT                            LOAD   ACTIVE SUB     DESCRIPTION
dhcpcd.service                  loaded active running DHCP Client
sshd.service                    loaded active running SSH Daemon
...

Here you can see some basic services like dhcpd and sshd . NixOS makes heavy usage of systemd, so we can use systemctl commands to find about the running services.

Nginx ¶

We're now inside NixOS territory. Since this blog is static html, we'll just need Nginx for publishing it.

So now we'll enable the Nginx service. You can see all of the NixOS systemd services and their options in Search NixOS options(or also by doing man configuration.nix).


# deploy.nix
let
  region = "us-east-2";
  accessKeyId = "default";
in {
  network.description = "stevechavez.xyz";

  resources = {
    ...
  };

  instance = { resources, ... }: {
    deployment = {
      targetEnv = "ec2";
      ec2 = {
        inherit region accessKeyId;
        instanceType             = "t2.nano";
        keyPair                  = resources.ec2KeyPairs.xyzKeyPair;
        elasticIPv4              = resources.elasticIPs.xyzIP;
        securityGroups           = [ resources.ec2SecurityGroups.xyzGroup ];
      };
    };

+   networking.firewall.allowedTCPPorts = [ 80 ];
+   services.nginx.enable = true;

  };
}


$ nixops deploy -d xyz

...
main.....> the following new units were started: nginx.service
main.....> activation finished successfully
xyz> deployment finished successfully

Here we started the nginx.service and also opened port 80 for http communication. We should now be able to get a 404 from Nginx, since we have no contents on the root url.


$ curl http://3.23.156.242/

<html>
<head><title>404 Not Found</title></head>
...

How is Nginx working here?

Let's have a look at the nginx.service file.


$ systemctl cat nginx.service

# /nix/store/f4wk1gnlx1cq598s77828msfgflnk8hx-unit-nginx.service/nginx.service
[Unit]
After=network.target
Description=Nginx Web Server

[Service]
...
ExecStart=/nix/store/6g4imgqjvk5anz4n0lmfbjby6f028b7n-nginx-1.16.1/bin/nginx -c '/nix/store/b1hymf3d1y3kqbjhimqj50bxjwmrhk08-nginx.conf' -p '/var/spool/nginx'
Group=nginx
User=nginx

In ExecStart, we can see the nginx command used to start the service. It uses the /var/spool/nginx² prefix, which contains the usual directories, including the logs:


$ nixops ssh -d xyz main ls /var/spool/nginx

client_body_temp
fastcgi_temp
logs
proxy_temp
scgi_temp
uwsgi_temp

The command also contains the full path of the nginx.conf. This contains a default config generated by NixOS, which we'll modify in the next step.

Blog ¶

To finish with the deployment, we'll now add this blog contents to Nginx, these are on a ./public directory.

To keep it simple, we'll use the services.nginx.config option. This corresponds to the contents of a regular nginx.conf file.


# deploy.nix
let
  region = "us-east-2";
  accessKeyId = "default";
in {
  network.description = "stevechavez.xyz";

  resources = {
    ...
  };

  instance = { resources, ... }: {
    deployment = {
      ...
    };
    networking.firewall.allowedTCPPorts = [ 80 ];

+   services.nginx = {
+     enable = true;
+     config = ''
+       events {}
+       http {
+         server {
+           listen 80;
+           root ${./public};
+         }
+       }
+     '';
+   };

  };
}


$ nixops deploy -d xyz

...
xyz> deployment finished successfully

We're now able to get the blog contents from the root url.


$ curl http://3.23.156.242/

<html>
<head>
<title>stevechavez.xyz</title>
...

Let's see what's the generated nginx.conf.


$ systemctl cat nginx.service | grep nginx.conf
ExecStart=/nix/store/6g4imgqjvk5anz4n0lmfbjby6f028b7n-nginx-1.16.1/bin/nginx -c '/nix/store/dvyyh6rh33vykbrypimmriih0cbihvki-nginx.conf' -p '/var/spool/nginx'

$ cat /nix/store/dvyyh6rh33vykbrypimmriih0cbihvki-nginx.conf


#/nix/store/dvyyh6rh33vykbrypimmriih0cbihvki-nginx.conf

pid /run/nginx/nginx.pid;
error_log stderr;
daemon off;
events {}
http {
  server {
    listen 80;
    root /nix/store/sq9hmkms4al81ykgd64c21v80fbjdpcn-public;
  }
}

The first three lines are added by default³. The rest of the config comes from our services.nginx.config.

The ${./public} fragment is string interpolation for adding the ./public relative path. This relative path gets translated⁴ to an absolute path in the Nix store.

Destroying ¶

By now the blog is live!

But to finish this post, we'll destroy it.


$ nixops destroy --confirm -d xyz

main.....> destroying EC2 machine... [shutting-down] [shutting-down] [shutting-down] [shutting-down] [shutting-down] [shutting-down] [shutting-down] [terminated]
xyzGroup..> deleting EC2 security group `xyz-group' ID `sg-09296f817f26cad4f'...
xyzIP.....> releasing elastic IP 3.23.156.242
xyzKeyPair> deleting EC2 key pair ‘charon-36d5282e-94b8-11ea-a0f2-047d7bf5e9c4-xyzKeyPair’...

$ nixops delete -d xyz

You can see the whole deploy.nix plus some goodies in this gist.

The location can be modified with the NIXOPS_STATE env var. For more details, check the man pages: man nixops. ↩︎
This can be modified with services.nginx.stateDir. To configure NixOS services, it's always a matter of looking through the existing options. ↩︎
This can be seen in nixpkgs. You can always look at nikpgs if you'd like to know the inner workings of a service. ↩︎
A hash is created out of the directory contents. For more details, see Nix Store Paths. ↩︎

Table of Contents