Unattended updates and upgrades

Aim

Implement an automatic update and upgrade mechanism both on existing and new hosts using the package unattended-upgrades and - in the case of updating only the package lists - configuration files in etc/apt/apt.conf.d.

Usage

First, clone the repos (including its submodules fsfe-system-hackers/inventory and jnv/ansible-role-unattended-upgrades)

git clone --recurse-submodules git@git.fsfe.org:fsfe-system-hackers/unattended-upgrades.git

Please note: It is advisable to regularly Update the inventory submodule in order to reflect the newest changes to the list of our hosts and the groups that they are in:

git submodule update --remote inventory

Apply upgrades automatically and reboot if necessary

TLDR

to reboot at 3am if required

ansible-playbook -i inventory/hosts -l reboot_3am auto_upgrade.yml

to reboot at 5am if required:

ansible-playbook -i inventory/hosts -l reboot_5am auto_upgrade.yml

The whole story

Most machines in our hosts should not require manual intervention for installing updates and should be automatically rebooted if the update requires it. Some machines, however should never reboot at the same time. Hence, there are two groups, namely reboot_3am and reboot_5am in our inventory of hosts. The first step for every update is an update of the package lists (apt-get update). As specified in the role configure timers, this happens at 1:00AM UTC (with a randomized delay of 30min) every night. This time can be specified using the variable apt_update_time.

The next step is then to run apt-get upgrade. This happens via the configuration of the package unattended-upgrades whose configuration will reside in /etc/apt.conf.d/50unattended-upgrades. The setup and configuration of unattended-upgrades is handled through the eponymous Ansible role by jnv. For details see on how this rule is used here, see the corresponding playbook file. If not otherwise specified via the variable apt-upgrade-time, upgrades are run at 2:00 UTC, again with a randomized delay of 30min. Subsequently, and only if required, the hosts are rebooted at either 3:00AM or 5:00AM depending on which group they are in.

Only download new packages (no upgrades, no reboot)

TLDR

ansible-playbook -i inventory/hosts -l update_only auto_update.yml

The whole story

Some hosts should not apply upgrades automatically. It's fine for them to download the packages that should be upgraded, but the run of apt-get upgrade itself should always be triggered by a human.

This is realised by configuring /etc/apt/apt.conf.d/20auto-updates-only as follows:

APT::Periodic::Unattended-Upgrade "0";
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "21";

To be sure, the apt-daily-upgrade.timer is disabled, while apt-daily.timer is left intact as specified above.

Resources

• Debian Wiki
• Debian Manual
• upstream of jnv's Ansible role at the heart of this repo
• An interesting thread on the issue on StackExchange
• a quick intro to unattended-upgrades

Troubleshooting

FATAL -> Failed to fork

This error seems to happen if too little RAM is available. However, there might also be other reasons, as this thread suggests.