More content and playbooks

* more readme.md content
* imported basic roles
* commented said basic roles
* trimmed down playbooks to be understandable
* update requirements.txt deps

Makefile

+install:
+	ansible-galaxy install -r requirements.yml

README.md

+# ansible-polytech-2023
+
+This is the repo that will serve as the support for the ASR practical work session of january. You should probably work in this directory and add code to it, it is going to be easier than creating a new ansible repository from scratch. You are also welcome to commit to this repository to checkpoint your work, as well as push it into any repository that polytech gives you access to.
+
+## Setup
+
+You will need to install a few things to get started, buckle up.
+
+### The virtualenv
+
+Since Ansible is written in python and we don't want to install it in the system, you will need to create a virtual environment. These are used to have your python stuff installed, without making them available system-wide, we are doing this to avoid polluting your lab machine with things that won't be used after today.
+
+To create the virtualenv you need to run the following:
+
+```
+$ python3 -m venv ~/.ansible-venv
+# then you want to "activate" the venv, you will need to do this for every new term you open
+$ . ~/.ansible-venv/bin/activate
+```
+
+:warning: you need to run `~/.ansible-venv/bin/activate` every time you want to open a new terminal and use ansible in it, otherwise it just won't work because the ansible binary won't be found.
+
+### Install ansible
+
+Install ansible via pip _after entering the venv_
+
+```
+$ pip install -r requirements.txt
+```
+
+At this point you should have ansible installed.
+
+### Install the docker role
+
+Install the docker role using ansible galaxy (ansible galaxy is a sort of package manager for ansible).
+
+```
+$ ansible-galaxy install -r requirements.yml
+```
+
+At this point you should be good to go!
+
+### Generate an SSH key if you don't have one already
+
+You'll need an SSH key if you don't have one already
+
+If `ssh-keygen` complains about the key already existing, just reuse the existing key in case someone else needs it.
+
+```
+$ ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 -P ''
+```
+
+:warning: In real life, don't use `-P ''` because it creates your SSH key without a passphrase, it is ok for this lab, not for real life.
+
+Lastly, look into `group_vars/all.yml`, go at the end of the file and add the created _public_ key in the root_user.default_root_keys (from ~/.ssh/id_ed25519.pub, or any other keys you created before hand). When this is done, add it to the `~/.ssh/authorized_keys` in the `root` home folder on every one of your virtual machines.
+
+### One more thing, update your inventory
+
+You can now update your `inventory` file by modifying it with your new values (hostnames and ip addresses for the machines you'll be working with).
+
+### Read the roles to understand how everything works !
+
+Ansible runs `playbooks`, which are collections of `roles` that in turn are a collection of `tasks`. `tasks` are instructions like "install this package", "copy this file", "create this directory", "install this service", "create this container" and so on and so forth.
+
+I have very much documented the example roles in `./roles` and I would _greatly_ encourage you to read them to understand how to do basic stuff in ansible such as copying a file, starting a service and so on. If you do not do that, you will be lost and won't understand anything that is coming at you.
+
+### Check everything works properly
+
+You should now be able to actually `run` ansible to execute the `base.yml` playbook.
+
+```
+$ ansible-playbook -vi inventory -l all base.yml
+```
+
+The `-i` flag specifies the inventory file to use, the `-l` file limits which hosts it applies to, either by hostname or group name, here we apply it to all the hosts.
+
+While you are at it I would recommend you install docker as well using the `docker.yml` playbook
+
+Good, now you are good to go !
+
+## Install a database server
+
+To deploy Synapse and Mastodon, you need to deploy a database server. We are going to use Postgres in this lab. You will for this need to use the `community.postgres_db` module for this. The `community.*` modules are modules written by the community and available to everyone, you will encounter similar modules when you will want to start deploying docker container!
+
+For more details to do this, I refer you to [this link](https://stribny.name/blog/ansible-postgresql/) which will walk you through how to deal with this.
+
+## Create a certificate for our deployments
+
+We need to use a certificate to secure HTTPS communication, both the Mastodon and Matrix protocols require it. This can be done manually for the moment and automated later, as it is not super straightforward. I refer you to the [documentation on certbot + nginx](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04) I would recommend that you create a `mastodon.<yourdomain>` and a `matrix.<yourdomain>` certificate, then back them up somewhere safe as Letsencrypt has pretty aggressive rate limiting with regard to certificate creations.

ansible.cfg

+[defaults]
+remote_user = root
+remote_port = 22
+host_key_checking = False
+#hash_behaviour = merge
+pipelining=True
+fact_path = facts.d
+gathering = implicit
+gather_subset = all
+inject_facts_as_vars = True

base.yml

+---
+- hosts: all
+  roles:
+    - role: root_user
+      tags: root_user
+    - role: base
+      tags: base
+    - role: vim
+      tags: vim

docker.yml

+---
+- hosts: all
+  roles:
+    - geerlingguy.docker

group_vars/all.yml

+---
+# Standard variables
+os: "{{ ansible_system|lower }}"
+arch: "{% if ansible_architecture == 'aarch64' %}arm64{% elif ansible_architecture == 'amd64' or ansible_architecture == 'x86_64' %}amd64{% elif ansible_architecture == 'armhf' %}armhf{% else %}{{ ansible_architecture }}{% endif %}"
+
+# Docker stuff
+docker_apt_ignore_key_error: false
+docker_install_compose: true
+docker_apt_arch: "{{ arch }}"
+
+root_user:
+  default_root_keys:
+    - name: thomas-ed25519
+      key: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPrw78OSJACq5MFXHrhAr2bPpnTNxwLE85mzij8gKmCs thomas@thonkpad

inventory

+[all]
+# Replace this with your hostname and IP address
+your-db-host.local ansible_ssh_host="1.2.3.4"
+your-docker-host.local ansible_ssh_host="5.6.7.8"
+
+[db]
+your-db-host.local ansible_ssh_host="1.2.3.4"
+
+[docker]
+your-docker-host.local ansible_ssh_host="5.6.7.8"

reboot.yml

+---
+- hosts: all
+  tasks:
+  - name: Reboot
+    reboot:
+      reboot_timeout: 3600
\ No newline at end of file

requirements.txt

+ansible==7.1.0
+ansible-core==2.14.1
+cffi==1.15.1
+cryptography==39.0.0
+Jinja2==3.1.2
+MarkupSafe==2.1.1
+packaging==23.0
+pycparser==2.21
+PyYAML==6.0
+resolvelib==0.8.1

requirements.yml

+---
+roles:
+  - src: geerlingguy.docker
+    version: 4.2.3
+collections:
+  - name: community.docker
+    version: 2.6.0

roles/base/tasks/main.yml

+---
+# welcome to tbe base.yml role. As you can see, this is a collection of
+# tasks, which in turn is basically a structure like this one
+# name: "user friendly name of your task"
+# a_module_name:
+#   a_module_parameter: value
+#   another_module_parameter: value2
+
+# this installs packages, here we use the `apt` module
+# because we are on a debian-based distribution. If we were
+# using something like Fedora for instance, we would have used
+# the `dnf` module. Then just pass as arguments the names of
+# the packages you want to install just as you would pass them
+# to `apt install`.
+- name: "Install wanted packages"
+  apt:
+    name:
+      - apt-transport-https
+      - arptables
+      - bash-completion
+      - bridge-utils
+      - ca-certificates
+      - cmake
+      - coreutils
+      - curl
+      - dnsutils
+      - ebtables
+      - file
+      - gawk
+      - git
+      - glusterfs-client
+      - glusterfs-server
+      - gnupg
+      - htop
+      - iftop
+      - ifstat
+      - iputils-ping
+      - iptables
+      - iproute2
+      - jq
+      - libsqlite3-dev
+      - libffi-dev
+      - libpython3-dev
+      - libssl-dev
+      - locales-all
+      - lsb-release
+      - lsof
+      - lvm2
+      - mdadm
+      - minicom
+      - mtr-tiny
+      - netcat
+      - net-tools
+      - ntp
+      - open-iscsi
+      - openssl
+      - p7zip-full
+      - pwgen
+      - sqlite3
+      - strace
+      - sudo
+      - sysstat
+      - telnet
+      - tcpdump
+      - tmux
+      - uuid-runtime
+      - unzip
+      - vim-nox
+      - wget
+      - wipe
+      - zip
+    # state: latest will update the package everytime the
+    # role is ran against a host
+    state: latest
+    # Update the cache before trying to update ?
+    # You generally want this because it's not updating itself
+    update_cache: true
+    # force update it if it's been updated for longer than an hour
+    cache_valid_time: 3600
+  register: apt_res
+  # if running apt fails, retry to do it up to 5 times then give up and cry in the corner
+  retries: 5
+  until: apt_res is success
+# Same as above, except you *remove* packages instead of installing them
+- name: "Remove unanted packages"
+  apt:
+    name:
+      - ntpdate
+    # note the value of `state` here
+    state: absent
+# This renders a template, the `inventory_hostname` is a variable
+# that is golbally available. It will map to the hostname you
+# assigned your host in the `inventory` file
+- name: "Hard set hostname"
+  template:
+    src: hostname.j2
+    dest: /etc/hostname
+    owner: root
+    group: root
+    # you need to put it in string mode, and you need to
+    # have a leading `0`, otherwise ansible is going to interpret it weird
+    # more info https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html#parameter-mode
+    mode: "0644"
+- name: "Setup motd"
+  template:
+    src: motd.j2
+    dest: /etc/motd
+    owner: root
+    group: root
+    mode: "0644"
+- name: "Setup hosts"
+  template:
+    src: hosts.j2
+    dest: /etc/hosts
+    owner: root
+    group: root
+    mode: "0644"
+# Note that you can also just `copy` files using the copy module:
+# https://docs.ansible.com/ansible/latest/collections/ansible/builtin/copy_module.html#examples
+# And finally you can create files and directories with the `file` module
+# https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html#examples

roles/base/templates/hostname.j2

+{{ inventory_hostname }}
\ No newline at end of file

roles/base/templates/hosts.j2

+127.0.0.1       localhost
+{{ ansible_default_ipv4["address"] }} {{ inventory_hostname }}
+

roles/base/templates/motd.j2

+            This is {{ ansible_fqdn }}
+
+System       : {{ ansible_distribution }} {{ ansible_distribution_version }} ({{ ansible_distribution_release }})
+
+Kernel       : {{ ansible_kernel }} {{ ansible_kernel_version }}
+CPU(s)       : {{ ansible_processor_cores }}
+RAM          : {{ ansible_memory_mb.real.total }}Mb
+Architecture : {{ ansible_architecture }}
+Address      : {{ ansible_default_ipv4.interface }} - {{ ansible_default_ipv4.address }} 

roles/ntp/files/ntp.conf

+# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help
+
+driftfile /var/lib/ntp/ntp.drift
+
+# Leap seconds definition provided by tzdata
+leapfile /usr/share/zoneinfo/leap-seconds.list
+
+# Enable this if you want statistics to be logged.
+#statsdir /var/log/ntpstats/
+
+statistics loopstats peerstats clockstats
+filegen loopstats file loopstats type day enable
+filegen peerstats file peerstats type day enable
+filegen clockstats file clockstats type day enable
+
+# Specify one or more NTP servers.
+
+# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
+# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
+# more information.
+pool 0.ubuntu.pool.ntp.org iburst
+pool 1.ubuntu.pool.ntp.org iburst
+pool 2.ubuntu.pool.ntp.org iburst
+pool 3.ubuntu.pool.ntp.org iburst
+
+# Use Ubuntu's ntp server as a fallback.
+pool ntp.ubuntu.com
+
+# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
+# details.  The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions>
+# might also be helpful.
+#
+# Note that "restrict" applies to both servers and clients, so a configuration
+# that might be intended to block requests from certain clients could also end
+# up blocking replies from your own upstream servers.
+
+# By default, exchange time with everybody, but don't allow configuration.
+restrict -4 default kod notrap nomodify nopeer noquery limited
+restrict -6 default kod notrap nomodify nopeer noquery limited
+
+# Local users may interrogate the ntp server more closely.
+restrict 127.0.0.1
+restrict ::1
+
+# Needed for adding pool entries
+restrict source notrap nomodify noquery
+
+# Clients from this (example!) subnet have unlimited access, but only if
+# cryptographically authenticated.
+#restrict 192.168.123.0 mask 255.255.255.0 notrust
+
+
+# If you want to provide time to your local subnet, change the next line.
+# (Again, the address is an example only.)
+#broadcast 192.168.123.255
+
+# If you want to listen to time broadcasts on your local subnet, de-comment the
+# next lines.  Please do this only if you trust everybody on the network!
+#disable auth
+#broadcastclient

roles/ntp/handlers/main.yml

+---
+# handlers are special tasks that you can invoke when something changes.
+# in this example you want to invoke this restart handler when the
+# configuration of the service changes for example.
+- name: "Restart ntp"
+  service:
+    name: ntp
+    state: restarted

roles/ntp/tasks/main.yml

+---
+- name: "Install ntp"
+  apt:
+    name: ntp
+    state: present
+- name: "Configure ntp"
+  copy:
+    src: ntp.conf
+    dest: /etc/ntp.conf
+    owner: root
+    group: root
+    mode: "644"
+  # notify is going to invoke the "Restart ntp" handler defined in
+  # the roles/ntp/handlers/main.yml file. Basically it says that
+  # you want to restart the ntp service every time the configuratio
+  # changes
+  notify: "Restart ntp"
+# Here you say that you want the NTP service to be restarted as well
+# as enabled on boot.
+- name: "ntp service"
+  service:
+    name: ntp
+    state: restarted
+    enabled: true

roles/root_user/tasks/main.yml

+---
+# Ensures the .ssh directory exists
+- name: "creates the .ssh root directory"
+  file:
+    path: "/root/.ssh"
+    state: directory
+    owner: root
+    group: root
+    mode: 0700
+- name: "Install root SSH keys"
+  template:
+    src: authorized_keys.j2
+    dest: /root/.ssh/authorized_keys
+    owner: root
+    group: root
+    mode: 0600
+# Delete users you don't need
+# respectively you can also *add* users
+# https://docs.ansible.com/ansible/latest/collections/ansible/builtin/user_module.html#ansible-collections-ansible-builtin-user-module
+- name: "Delete usual cloud users user"
+  user:
+    name: "{{ item }}"
+    state: absent
+    remove: true
+  with_items:
+    - pi
+    - admin
+    - ubuntu

roles/root_user/templates/authorized_keys.j2

+{% for key in root_user.default_root_keys %}
+# {{ key.name }}
+{{ key.key }}
+{% endfor %}

roles/vim/files/vimrc

+set nocompatible
+
+filetype on
+filetype plugin indent on
+syntax on
+
+set expandtab
+set tabstop=4
+set softtabstop=4
+set shiftwidth=4
+set smarttab
+set lazyredraw
+set showmatch
+set incsearch
+set hlsearch
+set number
+set ruler
+set autoindent
+set smartindent
+set wildmenu
+set laststatus=2
+set cursorline
+set showmode
+set backspace=indent,eol,start
+set mouse=a
+set background=dark
+set foldenable
+set foldlevelstart=10
+set foldmethod=indent
+
+nnoremap <F2> :set invpaste paste?<CR>
+set pastetoggle=<F2>
+set showmode
+
+let mapleader=","
+
+nnoremap <leader><space> :nohlsearch<CR>
+nnoremap <space> za
+map <Leader><Right> 10<C-w><<CR>
+map <Leader><Down> 10<C-w>-<CR>
+map <Leader><Up> 10<C-w>+<CR>
+map <Leader><Left> 10<C-w>><CR>
+map <Leader>= 10<C-w>=<CR>
+map <Leader>s :%s/\s\+$//<CR>
+
+nnoremap tt    :tabnew<CR>
+nnoremap to    :tabonly<CR>
+nnoremap tc    :tabclose<CR>
+nnoremap tn    :tabnext<CR>
+nnoremap tp    :tabprevious<CR>
+nnoremap vs    :vsplit