Merge pull request 'feat: add some docs with an overview of the build process' (#4870 ) from feat/overview-docs into master

Reviewed-on: #4870 Reviewed-by: Sofía Aritz <sofiaritz@fsfe.org>
feat: add links to next section
2025-03-10 12:40:03 +00:00 · 2025-03-10 12:38:55 +00:00 · 2025-03-10 12:38:55 +00:00
6 changed files with 95 additions and 1 deletions
--- a/README.md
+++ b/README.md
@ -13,7 +13,9 @@ This repository contains the source files of [fsfe.org](https://fsfe.org), pdfre
 ## Technical information
-Our web team has compiled some information about technology used for this website on the [Information for Webmasters](https://fsfe.org/contribute/web/) page.
+Our web team has compiled some information about technology used for this website on the [Information for Webmasters](https://fsfe.org/contribute/web/) page. This is mainly focused on page content.
 For information on how the build process works see [docs subfolder](./docs/overview.md)
 ## Structure
--- a/docs/overview.md
+++ b/docs/overview.md
@ -0,0 +1,38 @@
 # Overview
 This is to serve as a general overview of the build process, largely aimed at those who intent to contribute to the process.
 Firstly, the language of the build process is XML. Everything is based around XML, XHTML, XSL, with a few non XML extras the build process adds for features like choosing source files.
 ## Simple walkthrough
 Firstly a general case of how the build process works. The build process,
 1. Takes an XHTML source file, with name `<name>.<lang>.xhtml`
 2. If it exists in the same dir, loads `<name>.sources` and makes any data files specified in it available.
 3. This making available is done by loading the source XHTML, and specified data files, texts files for common strings into a new in memory XML.
 4. Selects an XSL file to process the in memory XML with. It will use `<name>.xsl` if available, else the first `default.xsl` it finds ascending through parent dirs.
 5. Processes the file, and puts the result into the output dir with name `<name>.<lang>.<html>`.
 6. If the file name matches its parent directory name, IE it is in `<name>/<name>.<lang>.xhtml`, then generate an index symlink in the output dir.
 7. Generate symlinks from `<name>.<lang>.html` to `<name>.html.<lang>`, for [Apache Multiviews](https://httpd.apache.org/docs/current/content-negotiation.html) (how we handle choosing pages based on user browser language)
 8. If the site has been staged, which occurs when manually specified, using multiple targets, or using ssh targets. In this cases the build process will have built the sites to a local directory, and will then copy them to each of the specified targets.
 ## General details
 The website uses incremental caching. These cache files are generally hidden files, and should be `gitignored`. For exactly what are cache files see the `gitignore` in the repo root. The build process determines what must be updated based on the file modification time of dependencies of any file.
 Care has been taken to make sure that files are rebuilt only when required, and are always rebuild when required. But changes to fundamentals may not propagate correctly and may require a manually triggered full rebuild, or waiting till the nightly one on prod.
 ## Phases
 For details on the phases and exact implementation please examine the codebase
 The build process can be conceptually divided into four phases: Phases 0-3
 ### [phase0](./phase0.md)
 ### [phase1](./phase1.md)
 ### [phase2](./phase2.md)
 ### [phase3](./phase3.md)
--- a/docs/phase0.md
+++ b/docs/phase0.md
@ -0,0 +1,7 @@
 # Phase 0
 Parse arguments, apply any special cases from arguments such as updating the repo using git, cleaning cache from old builds for a full rebuild, etc.
 Once this phase is over our dependencies are available, arguments parsed and ready to start the next phase.
 And now, [phase1](./phase1.md)
--- a/docs/phase1.md
+++ b/docs/phase1.md
@ -0,0 +1,19 @@
 # Phase 1
 Phase 1 is arguably the most complicated phase.
 This phase never modifies the output directory, it concerns itself only with generating files in the source tree that are then used in phase 2.
 It handles the generation of files not generated with an XSL processor (search indices, compiling CSS, etc) and ensuring the dependencies of each XHTML are present and up to date.
 Phase 1 goes through XSL stylesheets, and updates the modification time of those whose parents have been updated, so that only the stylesheet an XHTML file depends on directly depends on modification times need be considered.
 It runs the subdirectory scripts, which perform custom actions required by those directories. For example, the news and events subdirectory in the main site, use subdirectory scripts to generate indices based on a template for every year in the archives.
 It also generates the paths for global texts files, for storing commonly used strings based on if that text exists or else it falls back to English.
 It does a few other things, as well, for an exhaustive list please examine the codebase.
 After phase 1 is complete we can be reasonably sure that dependencies for phase 2 etc are in place, and with timestamps that phase2 can depend on to be useful.
 And now, [phase2](./phase2.md)
--- a/docs/phase2.md
+++ b/docs/phase2.md
@ -0,0 +1,19 @@
 # Phase 2
 After all the XHTML files and their dependencies have been nicely lined up and prepared in phase 1, phase 2 begins
 It will
 1. Takes an XHTML source file, with name `<name>.<lang>.xhtml`
 2. If it exists in the same dir, loads `<name>.sources` and makes any data files specified in it available.
 3. This making available is done by loading the source XHTML, and specified data files, texts files for common strings into a new in memory XML.
 4. Selects an XSL file to process the in memory XML with. It will use `<name>.xsl` if available, else the first `default.xsl` it finds ascending through parent dirs.
 5. Processes the file, and puts the result into the output dir with name `<name>.<lang>.<html>`.
 6. If the file name matches its parent directory name, IE it is in `<name>/<name>.<lang>.xhtml`, then generate an index symlink in the output dir.
 7. Generate symlinks from `<name>.<lang>.html` to `<name>.html.<lang>`, for [Apache Multiviews](https://httpd.apache.org/docs/current/content-negotiation.html) (how we handle choosing pages based on user browser language)
 8. Using some somewhat advanced XSL wizardry, Phase 2 is also able to build RSS and ICS files from XML data. The process of selecting stylesheets etc is the same as for XHTML>
 9. Copies over any static files that are needed, like images, PDFs etc.
 After phase 2 is over we have a copy of all sites built. If the site was not staged then it is in the target directory if it was staged, then it is in the staging directory.
 And now, [phase3](./phase3.md)
--- a/docs/phase3.md
+++ b/docs/phase3.md
@ -0,0 +1,9 @@
 # Phase 3
 This final phase relates to finishing up the build
 If the build was staged then this is the phase where the built result gets copied to the target(s).
 And if specified the resulting build is served over localhost for rapid debugging.
 And finished!
Author	SHA1	Message	Date
delliott	2f83618f35	Merge pull request 'feat: add some docs with an overview of the build process' (#4870 ) from feat/overview-docs into master All checks were successful continuous-integration/drone/push Build is passing Details Reviewed-on: #4870 Reviewed-by: Sofía Aritz <sofiaritz@fsfe.org>	2025-03-10 12:40:03 +00:00
Darragh Elliott	c982f6294a	feat: add links to next section All checks were successful continuous-integration/drone/pr Build is passing Details	2025-03-10 12:38:55 +00:00
Darragh Elliott	e87d0838e1	feat: add some docs with an overview of the build process	2025-03-10 12:38:55 +00:00