DevOps #3 - Docker RootLess

DevOps #2 Au secours, mon projet Docker est en production

Docker Rootless

rootless

Le mode rootless permet une simulation pour le démon docker ainsi que pour le conteneur d’un utilisateur root. Il utilise les namespaces sur Linux pour réaliser ce subterfuge.

A ne pas confondre avec :

sudo docker

usermod -aG docker mcalves

docker run --user 42

Pourquoi se mettre en rootless en production ?

  • Le mode Rootless permet d’atténuer les zones d’attaques
  • Accéder à des fichiers appartenant à d’autres utilisateurs.
  • Modifier le microprogramme ou le noyau
  • L’usurpation d’identité de l’ARP
  • Votre politique de sécurité l’impose

Initialisation du Workshop

Dépendances du workhsop

  • Un serveur de virtualisation (Hyper-V, VirtualBox, VMWare WorkStation)
  • Une VM Debian 10

Vous avez besoin d’un serveur de virtualisation si vous ne possèdez pas de machine Linux sous Debian 10.

Architecture du Workshop

image.png

L’ensemble des services présent sur la VM seront exécuter sans privilège administrateur afin de réduire le risque d’attaque en limitant les droits des services.

Mise en place du Workshop

Créer, installer, configurer votre VM debian sur Hyper-V : Tuto

Déploiement du service de Reverse Proxy (NGINX)

Ce service sera exécuter dans droits d’accès administrateur.

Installation

sudo apt-get -y update 
sudo apt-get - install nginx

Configuration

sudo chown -R /etc/nginx mcalves:mcalves
sudo chown -R /var/log/nginx mcalves:mcalves
mkdir $HOME/nginx/ && touch $HOME/nginx/nginx.pid

vim /etc/nginx/nginx.conf

worker_processes auto;
pid /home/mcalves/nginx/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
error_log /var/log/nginx/error.log;
worker_rlimit_nofile 8192;

events {
        worker_connections 4096;
        # multi_accept on;
}


http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        proxy_read_timeout 600s;
        proxy_http_version 1.1;
        proxy_buffering off;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

  # allow large uploads of files
  client_max_body_size 1G;

  ##
  # Reverse Proxy
  ##
  server {
    listen 8080;
    server_name repo.example.com;
    location / {
      proxy_set_header  Host              $http_host;
      proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
      proxy_pass http://127.0.0.1:8081/;
   }
  }

  server {
    listen 8080;
    server_name gitlab.example.com;
    location / {
      proxy_set_header  Host              $http_host;
      proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
      proxy_pass http://127.0.0.1:9080/;
  }
  }
        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

Dans cet exemple on retrouve des redirections vers des services comme Gitlab ou Nexus. A modifier pour votre service.

vim /etc/nginx/sites-available/default

server {
    listen 8080 default_server;
    listen[::]:8080 default_server;
...
}

Configuration de la rotation des messages de logs

sudo vim /etc/logrotate.d/nginx

{
...
        create 0640 mcalves mcalves
...
}

Configuration de systemd

sudo echo "mcalves ALL=(ALL) NOPASSWD:/usr/sbin/nginx" > /etc/sudoers.d/mcalves
sudo cp /lib/systemd/system/nginx.service /etc/systemd/system

sudo vim /lib/systemd/system/nginx.service

[Unit]
Description=A high performance web server and a reverse proxy server
Documentation=man:nginx(8)
After=network.target nss-lookup.target

[Service]
Type=forking
PIDFile=/home/mcalves/nginx/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /home/mcalves/nginx/nginx.pid
TimeoutStopSec=5
KillMode=mixed
User=mcalves
Group=mcalves
[Install]
WantedBy=multi-user.target

sudo systemctl enable nginx
sudo systemctl start nginx
sudo systemctl status nginx

ps aux |grep nginx

mcalves@ats-linux-03:~/gitlab$ ps aux | grep nginx
mcalves    837  0.0  0.0  10392   852 ?        Ss   10:45   0:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
mcalves    838  0.0  0.0  12168  3228 ?        S    10:45   0:00 nginx: worker process
mcalves    839  0.0  0.0  12168  3228 ?        S    10:45   0:00 nginx: worker process
mcalves    840  0.0  0.0  12168  3228 ?        S    10:45   0:00 nginx: worker process

Déploiement du service Docker Rootless

Scénario Katacoda du mode Rootless Tuto

Création d’un compte de service Linux

groupadd pic --gid 65536
adduser pic --uid 231072 --gid 65536
echo "pic:100000:65536" >> /etc/subuid
echo "pic:100000:65536" >> /etc/subgid
echo "kernel.unprivileged_userns_clone=1" >> /etc/sysctl.d/99-sysctl.conf
sysctl --system
su pic
id -u
whoami
grep ^$(whoami): /etc/subuid
grep ^$(whoami): /etc/subgid

Installation de Docker en mode Rootless

curl -fsSL https://get.docker.com/rootless | sh
echo "export XDG_RUNTIME_DIR=/run/user/$(id -u)" $HOME/.bashrc
echo "export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/docker.sock" $HOME/.bashrc
echo "export PATH=/home/mcalves/bin:$PATH" $HOME/.bashrc
echo "export COMPOSE_HTTP_TIMEOUT=200" $HOME/.bashrc
source $HOME/.bashrc

Configuration du systemd

mkdir -p $HOME/.config/systemd/user/
vim $HOME/.config/systemd/user/docker.service
[Unit]
Description=Docker Application Container Engine (Rootless)
Documentation=https://docs.docker.com

[Service]
Environment=PATH=$HOME/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ExecStart=$HOME/bin/dockerd-rootless.sh --experimental --storage-driver vfs
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
Type=simple

[Install]
WantedBy=default.target

systemctl --user daemon-reload
systemctl --user start docker
systemctl --user status docker

Déploiement du service Gitlab

vim docker-compose.yml

version: "3.8"
services:
  gitlab:
    image: gitlab/gitlab-ce:12.10.3-ce.0
    restart: unless-stopped
    hostname: gitlab.example.com
    container_name: 'gitlab'
    ports:
      - "9022:22"
      - "9080:80"
    volumes:
      - gitlab-data:/var/opt/gitlab
      - gitlab-logs:/var/log/gitlab
      - gitlab-config:/etc/gitlab
    environment:
       GITLAB_OMNIBUS_CONFIG: |
         external_url 'http://IP_HOSTNAME'
          # Add any other gitlab.rb configuration here, each on its own line

volumes:
  gitlab-data:
  gitlab-logs:
  gitlab-config:

docker-compose up -d

Résultat attendu :

docker ps

mcalves@ats-linux-03:~/gitlab$ docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS                  PORTS                                                 NAMES
868af07fe3d3        gitlab/gitlab-ce:12.10.3-ce.0   "/assets/wrapper"        16 hours ago        Up 16 hours (healthy)   443/tcp, 0.0.0.0:9022->22/tcp, 0.0.0.0:9080->80/tcp   gitlab

Résultat attendu :

ps aux | grep docker

root@ats-linux-03:~# ps aux | grep docker
mcalves   1411  0.0  0.0 110384 12160 ?        Sl   05:06   0:00 rootlesskit --net=vpnkit --mtu=1500 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run /home/mcalves/bin/dockerd-rootless.sh --experimental --storage-driver vfs
mcalves   1420  0.0  0.0 110384 12888 ?        Sl   05:06   0:00 /proc/self/exe --net=vpnkit --mtu=1500 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run /home/mcalves/bin/dockerd-rootless.sh --experimental --storage-driver vfs
mcalves   1455  1.2  0.9 940276 145860 ?       Sl   05:06   2:39 dockerd --experimental --storage-driver vfs
mcalves   1468  0.3  0.2 702860 36004 ?        Ssl  05:06   0:42 containerd --config /run/user/1000/docker/containerd/containerd.toml --log-level info
root      3259  0.0  0.0   6224   884 pts/1    S+   08:43   0:00 grep docker
mcalves  14197  0.0  0.0 106632  8076 ?        Sl   06:12   0:00 /home/mcalves/bin/rootlesskit-docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 8081 -container-ip 172.27.0.2 -container-port 8081
mcalves  14202  0.0  0.0 103324  5104 ?        Sl   06:12   0:00 docker-proxy -container-ip 172.27.0.2 -container-port 8081 -host-ip 127.0.0.1 -host-port 8081 -proto tcp
mcalves  14211  0.0  0.0 107700  8168 ?        Sl   06:12   0:06 containerd-shim -namespace moby -workdir /home/mcalves/.local/share/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/e2aaecf2a3a7b332e9caeb14b565f13309abb7cd618f3f9e0fb6adb1e36b41a3 -address /run/user/1000/docker/containerd/containerd.sock -containerd-binary /home/mcalves/bin/containerd -runtime-root /run/user/1000/docker/runtime-runc
mcalves  20096  0.0  0.0 108296  7836 ?        Sl   06:50   0:00 /home/mcalves/bin/rootlesskit-docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 9080 -container-ip 172.23.0.2 -container-port 80
mcalves  20101  0.0  0.0 103580  5104 ?        Sl   06:50   0:00 docker-proxy -container-ip 172.23.0.2 -container-port 80 -host-ip 127.0.0.1 -host-port 9080 -proto tcp
mcalves  20118  0.0  0.0 106632  8080 ?        Sl   06:50   0:00 /home/mcalves/bin/rootlesskit-docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 9022 -container-ip 172.23.0.2 -container-port 22
mcalves  20123  0.0  0.0 103324  5044 ?        Sl   06:50   0:00 docker-proxy -container-ip 172.23.0.2 -container-port 22 -host-ip 127.0.0.1 -host-port 9022 -proto tcp
mcalves  20132  0.0  0.0 109108 10156 ?        Sl   06:50   0:01 containerd-shim -namespace moby -workdir /home/mcalves/.local/share/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/868af07fe3d39afb1e2bb12a5bb9b16c0fbb4fe69667598131b54a1a8094395f -address /run/user/1000/docker/containerd/containerd.sock -containerd-binary /home/mcalves/bin/containerd -runtime-root /run/user/1000/docker/runtime-runc

Déploiement du service Nexus

vim docker-compose.yml

version: "3.8"
services:
  nexus:
    image: sonatype/nexus3:3.23.0
    restart: unless-stopped
    hostname: nexus.example.com
    container_name: 'nexus'
    ports:
      - "8081:8081"
    volumes:
      - nexus-data:/nexus-data

volumes:
  nexus-data:

mcalves@ats-linux-03:~/nexus$ docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS                       PORTS                                                 NAMES
c0cefb0dbf48        sonatype/nexus3:3.23.0          "sh -c ${SONATYPE_DI…"   25 hours ago        Up About an hour             0.0.0.0:8081->8081/tcp                                nexus

Modifier son fichier /etc/hosts et ajoutée le nom de domaine avec l’IP du serveur, par exemple :

192.168.253.21  repo.example.com
192.168.253.21  gitlab.example.com