Matomo website analytics
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Max Mehl d87aeaffff
import logs from second webserver, too
2 weeks ago
LICENSES make REUSE compliant 2 years ago
.drone.yml fixed order 3 months ago
.gitignore make REUSE compliant 2 years ago
Dockerfile-mariadb use static major version of mariadb 1 year ago
Dockerfile-matomo set logfile permissions to avoid cron errors 2 years ago
README.md restrict access of matomo get_logs.sh user 6 months ago
clean-matomo-tables.sh make REUSE compliant 2 years ago
cron.txt import logs from second webserver, too 2 weeks ago
cron.txt.license make REUSE compliant 2 years ago
docker-compose.yml migrate matomo to cont2 3 months ago
get_logs.sh fix issue with remote sed 3 months ago
host_options.ini make REUSE compliant 2 years ago
id_rsa.pub change SSH key 2 years ago
id_rsa.pub.license make REUSE compliant 2 years ago
rp_hosts.csv make REUSE compliant 2 years ago

README.md

REUSE status Build Status

FSFE Matomo/Piwik

This service serves as the single point inside the FSFE to gather web statistics for different domains and evaluate them.

Because we do not like JavaScript that much, we import the webserver logs directly into Matomo.

We use the standard Docker image for Matomo but with a few modifications, mostly concerning the automated log import.

Log import

Modes

The normal mode in get_logs.sh connects via SSH to download the most recently rotated Apache log.

The docker mode instead reads the logs of the reverse proxy which is in front of all our Docker containers that run publicly facing web services. For all containers defined in rp_hosts.csv, it scrapes the logs and imports them.

You can run get_logs.sh -h to read more about the script's usage.

Host options

Because logs for some sites need special treatment before to be more useful or require certain import options, there is the file host_options.ini in which you can define per page a) one shell command to run over the log, and b) options for Matomo's import_logs.py.

Scheduling

The cron.txt file is appended to /etc/crontab in the matomo container and controls when imports are scheduled for which pages.

Server preparation

Normal (Apache)

To restrict rsync's access to only the log files, do:

  1. cp /usr/share/doc/rsync/scripts/rrsync /usr/local/bin/
  2. chmod +x /usr/local/bin/rrsync
  3. Add in root's authorized_keys file: command="/usr/local/bin/rrsync -ro /var/log/apache2/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa ...
  4. The Apache webserver file indicated in cron.txt is relative to the path declared in command=

Docker (reverse proxy)

  1. In the Docker daemon's user authorized_keys file: command="docker logs ${SSH_ORIGINAL_COMMAND#*docker logs}",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa...

Clean Matomo site

In the matomo-mariadb container, you can run clean-matomo-tables.sh to clean the stats for one or all sites. This is useful if the recording went mad during your tests. But remember that this is a complete reset!

Environment variables

Docker-compose wants to have the following environment variables:

  • SSH_KEY: the SSH key used to connect to the individual hosts via SSH
  • MYSQL_DATABASE: the name of the database
  • MYSQL_ROOT_PASSWORD: root password for the DB
  • MYSQL_USER: user for Matomo
  • MYSQL_PASSWORD: password for the Matomo user

The other variables are rather standard for the FSFE's usual setup.

License

All custom scripts and files in this repo are licensed under GPL-3.0-or-later.