FSFE Community Database

Build status

Staging
Production

Background

The FSFE Community Database is a combined tool for donation management, administration of accounts for FSFE's services, and opt-in based distribution of information emails.

FSFE Database Authentication Server

All relevant data for these purposes is stored in a PostgreSQL database on the one hand and FSFE's LDAP server on the other hand. Access to that data is encapsulated by the FSFE Community Database Backend, which provides both a set of commandline tools for manual administration tasks, and a protected RESTful(ish) network API to be used by other components.

The FSFE Community Database Backend essentially consists of three parts:

1. An Object-Relational Mapper (ORM) which turns database records into Python objects and adds all the methods necessary to perform the tasks the FSFE Community Database is built for. This layer is the core of the whole structure. It is built with SQLAlchemy and resides in fsfe_cd_back/data.

2. A resource server with a REST-like API to allow data access for the other components of fsfe-cd. All its access to the actual data goes through the ORM. The resource server is implemented with Flask and can be found in fsfe_cd_back/server.

3. A set of command line tools for manual administration tasks, most of which are only a thin wrapper around methods implemented in the ORM. They are collected in the bin directory.

FSFE Database Authentication Server

The FSFE Community Database Authentication Server is an OpenID Connect Provider running on https://id.fsfe.org which authenticates a user against the LDAP server and then issues an access token which provides access to the authenticated user's (and only the authenticated user's) data though fsfe-cd-back's API.

Currently, this is an incomplete implementation of the OpenID Connect standard, providing only those functions required by fsfe-cd-back and fsfe-cd-front.

Implementation is based on pyoidc with a thin Flask wrapper for the URL routing.

Authentication is possible with two distinct methods:

1. Authentication with username and password. Instead of the username, an email address can be given, in which case the corresponding username will be looked up via fsfe-cd-back. Username and password will be verified by a direct query to the LDAP server.

2. If only username or email address is given, but no password, then the user receives an email with a login token encapsulated in a link which will allow a one-time login. This method is not only helpful for resetting a forgotten password, it also allows users to log in who have not set a username.

FSFE Community Database Frontend

Finally, the FSFE Community Database Frontend is a web application through which the people stored in the database can manage their own data at https://my.fsfe.org. It sends users to fsfe-cd-auth to log in and then uses the user-bound access token returned from there to access the user's data through fsfe-cd-back.

fsfe-cd-front is a web application built with Flask. It provides a number of endpoints (called "views" in Flask) of the following categories:

1. Endpoints for interaction with the actual user: registering as a new user (register.py and donate.py), paying online (payonline.py), logging in and out (login.py), and viewing as well as updating the personal settings (settings.py).

2. Endpoints for handling predefined commands which can be executed by simply following a prepared link (command.py).

3. Endpoints for the automatic registration of incoming payments through PayPal or ConCardis (register_payment.py)

Translations

Development

The development of the FSFE Community Database is based on docker and docker-compose. Please note that this has been more tested against a rootless Docker daemon and since we run this type of daemon in both staging and production environments, it is recommended you have the same daemon running on your local machine. Here are Docker's installation instructions. Along with a text editor, these are the only two hard dependencies you need to have installed in order to start developing.

If you want to contribute, you also need pipenv and pre-commit installed locally. After installing pre-commit, you need to run

pre-commit install

in order to set up the check to run before every commit.

Start the development containers

Once you have installed everything, run

make dev

to start all three modules, i.e. back, front and auth in unison. The corresponding directories in this repository are mounted inside these containers so you can start changing the code in one of them, and the apps will update automatically during runtime as they all run the built-in Flask debugging server. Once you're done with your changes run

make qc.all

before committing as the commands run by this rule will need to complete without errors in order to pass through our CI system.

If you want to find out what other commands are available via the Makefile, run

make

to display a small manual page.

Deployment

Deployment is handled by Drone CI and can be configured by editing .drone.yml.

Staging

Pushing to the master branch triggers a deployment of the staging environment on cont2-noris that can be tested at https://staging.my.fsfe.org. The relevant Docker Comopose file is docker-compose.staging.yml

Production

Pushing to the protected production branch triggers a deployment of the production environment on cont1-noris that can be accessed via https://my.fsfe.org. The relevant Docker Comopose file is docker-compose.production.yml

Secrets

The following secrets are managed in drone:

Name	Description	Requirement	Component
jwt_secret	Shared secret for signed JSON Web Tokens used for authentication	Must match the secret with the same name for fsfe-cd-front	auth
bank_user	User id to log into the banking interface with aqbanking-cli	Must match the user id assigned by the bank	back
cmd_passphrase	Passphrase to verify URL-based commands	(no requirement, can be randomly generated)	back
postgres_password	Password for the PostgreSQL database running on seaborg.fsfeurope.org	Must match the password configured on the PostgreSQL server	back
ldap_password	Password for the LDAP server running at berzelius.fsfeurope.org	Must match the password configured on the LDAP server	back
concardis_passphrase	Passphrase to verify the ConCardis payment notifications	Must match the SHA-OUT passphrase defined in the ConCardis backoffice portal	back
bank_user	User id to log into the banking interface with aqbanking-cli	Must match the user id assigned by the bank	back
freescout_api_key	API Key needed for the interactions with https://helpdesk.fsfe.org	None	back
secret_key	Secret key used by flask for various purposes	Can be arbitrarily assigned, but should remain constant over server rebuilds	front
jwt_secret	Shared secret for signed JSON Web Tokens used for authentication	Must match the secret with the same name for fsfe-cd-auth	front
concardis_sha_passphrase	Passphrase to sign the ConCardis payment parameters	Must match the SHA-IN passphrase defined in the ConCardis backoffice portal	front

Maintainers

@reinhard @max @linus

Contributing

Small note: If editing the README, please conform to the standard-readme specification.

License

AGPL-3.0-or-later © 2021 FSFE System Hackers