FSFE/fsfe-website
25
31
Fork 92
Code Issues 65 Pull Requests 5 Wiki Activity

feat: add some docs with an overview of the build process

Browse Source
This commit is contained in:
Darragh Elliott 2025-03-04 15:54:40 +00:00
parent 199a29c78c
commit e87d0838e1
6 changed files with 87 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -13,7 +13,9 @@ This repository contains the source files of [fsfe.org](https://fsfe.org), pdfre
## Technical information
Our web team has compiled some information about technology used for this website on the [Information for Webmasters](https://fsfe.org/contribute/web/) page.
Our web team has compiled some information about technology used for this website on the [Information for Webmasters](https://fsfe.org/contribute/web/) page. This is mainly focused on page content.
For information on how the build process works see [docs subfolder](./docs/overview.md)
## Structure

38
docs/overview.md Normal file
View File

@ -0,0 +1,38 @@
# Overview
This is to serve as a general overview of the build process, largely aimed at those who intent to contribute to the process.
Firstly, the language of the build process is XML. Everything is based around XML, XHTML, XSL, with a few non XML extras the build process adds for features like choosing source files.
## Simple walkthrough
Firstly a general case of how the build process works. The build process,
1. Takes an XHTML source file, with name `<name>.<lang>.xhtml`
2. If it exists in the same dir, loads `<name>.sources` and makes any data files specified in it available.
3. This making available is done by loading the source XHTML, and specified data files, texts files for common strings into a new in memory XML.
4. Selects an XSL file to process the in memory XML with. It will use `<name>.xsl` if available, else the first `default.xsl` it finds ascending through parent dirs.
5. Processes the file, and puts the result into the output dir with name `<name>.<lang>.<html>`.
6. If the file name matches its parent directory name, IE it is in `<name>/<name>.<lang>.xhtml`, then generate an index symlink in the output dir.
7. Generate symlinks from `<name>.<lang>.html` to `<name>.html.<lang>`, for [Apache Multiviews](https://httpd.apache.org/docs/current/content-negotiation.html) (how we handle choosing pages based on user browser language)
8. If the site has been staged, which occurs when manually specified, using multiple targets, or using ssh targets. In this cases the build process will have built the sites to a local directory, and will then copy them to each of the specified targets.
## General details
The website uses incremental caching. These cache files are generally hidden files, and should be `gitignored`. For exactly what are cache files see the `gitignore` in the repo root. The build process determines what must be updated based on the file modification time of dependencies of any file.
Care has been taken to make sure that files are rebuilt only when required, and are always rebuild when required. But changes to fundamentals may not propagate correctly and may require a manually triggered full rebuild, or waiting till the nightly one on prod.
## Phases
For details on the phases and exact implementation please examine the codebase
The build process can be conceptually divided into four phases: Phases 0-3
### [phase0](./phase0.md)
### [phase1](./phase1.md)
### [phase2](./phase2.md)
### [phase3](./phase3.md)

5
docs/phase0.md Normal file
View File

@ -0,0 +1,5 @@
# Phase 0
Parse arguments, apply any special cases from arguments such as updating the repo using git, cleaning cache from old builds for a full rebuild, etc.
Once this phase is over our dependencies are available, arguments parsed and ready to start the next phase

17
docs/phase1.md Normal file
View File

@ -0,0 +1,17 @@
# Phase 1
Phase 1 is arguably the most complicated phase.
This phase never modifies the output directory, it concerns itself only with generating files in the source tree that are then used in phase 2.
It handles the generation of files not generated with an XSL processor (search indices, compiling CSS, etc) and ensuring the dependencies of each XHTML are present and up to date.
Phase 1 goes through XSL stylesheets, and updates the modification time of those whose parents have been updated, so that only the stylesheet an XHTML file depends on directly depends on modification times need be considered.
It runs the subdirectory scripts, which perform custom actions required by those directories. For example, the news and events subdirectory in the main site, use subdirectory scripts to generate indices based on a template for every year in the archives.
It also generates the paths for global texts files, for storing commonly used strings based on if that text exists or else it falls back to English.
It does a few other things, as well, for an exhaustive list please examine the codebase.
After phase 1 is complete we can be reasonably sure that dependencies for phase 2 etc are in place, and with timestamps that phase2 can depend on to be useful

17
docs/phase2.md Normal file
View File

@ -0,0 +1,17 @@
# Phase 2
After all the XHTML files and their dependencies have been nicely lined up and prepared in phase 1, phase 2 begins
It will
1. Takes an XHTML source file, with name `<name>.<lang>.xhtml`
2. If it exists in the same dir, loads `<name>.sources` and makes any data files specified in it available.
3. This making available is done by loading the source XHTML, and specified data files, texts files for common strings into a new in memory XML.
4. Selects an XSL file to process the in memory XML with. It will use `<name>.xsl` if available, else the first `default.xsl` it finds ascending through parent dirs.
5. Processes the file, and puts the result into the output dir with name `<name>.<lang>.<html>`.
6. If the file name matches its parent directory name, IE it is in `<name>/<name>.<lang>.xhtml`, then generate an index symlink in the output dir.
7. Generate symlinks from `<name>.<lang>.html` to `<name>.html.<lang>`, for [Apache Multiviews](https://httpd.apache.org/docs/current/content-negotiation.html) (how we handle choosing pages based on user browser language)
8. Using some somewhat advanced XSL wizardry, Phase 2 is also able to build RSS and ICS files from XML data. The process of selecting stylesheets etc is the same as for XHTML>
9. Copies over any static files that are needed, like images, PDFs etc.
After phase 2 is over we have a copy of all sites built. If the site was not staged then it is in the target directory if it was staged, then it is in the staging directory.

7
docs/phase3.md Normal file
View File

@ -0,0 +1,7 @@
# Phase 3
This final phase relates to finishing up the build
If the build was staged then this is the phase where the built result gets copied to the target(s).
And if specified the resulting build is served over localhost for rapid debugging.