FSFE Matomo/Piwik

This service serves as the single point inside the FSFE to gather web statistics for different domains and evaluate them.

Because we do not like JavaScript that much, we import the webserver logs directly into Matomo.

We use the standard Docker image for Matomo but with a few modifications, mostly concerning the automated log import.

Log import

Modes

The normal mode in get_logs.sh connects via SSH to download the most recently rotated Apache log.

The docker mode instead reads the logs of the reverse proxy which is in front of all our Docker containers that run publicly facing web services. For all containers defined in rp_hosts.csv, it scrapes the logs and imports them.

You can run get_logs.sh -h to read more about the script's usage.

Host options

Because logs for some sites need special treatment before to be more useful or require certain import options, there is the file host_options.ini in which you can define per page a) one shell command to run over the log, and b) options for Matomo's import_logs.py.

Scheduling

The cron.txt file is appended to /etc/crontab in the matomo container and controls when imports are scheduled for which pages.

Server preparation

Normal (Apache)

To restrict rsync's access to only the log files, provide rsync's rrsync:

1. Put rrsync to /usr/local/bin/
2. chmod +x /usr/local/bin/rrsync
3. Add in root's authorized_keys file: command="/usr/local/bin/rrsync -ro /var/log/apache2/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa ...
4. The Apache webserver file indicated in cron.txt is relative to the path declared in command=

Docker (reverse proxy)

1. In the Docker daemon's user authorized_keys file: command="docker logs ${SSH_ORIGINAL_COMMAND#*docker logs}",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa...

Clean Matomo site

In the matomo-mariadb container, you can run clean-matomo-tables.sh to clean the stats for one or all sites. This is useful if the recording went mad during your tests. But remember that this is a complete reset!

Environment variables

Docker-compose wants to have the following environment variables:

• SSH_KEY: the SSH key used to connect to the individual hosts via SSH
• MYSQL_DATABASE: the name of the database
• MYSQL_ROOT_PASSWORD: root password for the DB
• MYSQL_USER: user for Matomo
• MYSQL_PASSWORD: password for the Matomo user

The other variables are rather standard for the FSFE's usual setup.

License

All custom scripts and files in this repo are licensed under GPL-3.0-or-later.