feat: build by site #5031

delliott · 2025-05-17T18:16:20Z

delliott commented

2025-05-17 18:16:20 +00:00

phase 1 and 2 now by site folder

This allows for us building each site in a different set of languages, wheres before we built every site in every language any site was in.

At the moment this will build each site in every language that there is at least one file of that language in the site.

So if a site has on file in lang AA, it will be built in lang AA. But if it has no files, it will not be built in that lang.

This is a performance enhancement, will do benchmarks later.

Some more complex heuristics for when we do/do not use a language for a file are being discussed in #4601 .

phase 1 and 2 now by site folder This allows for us building each site in a different set of languages, wheres before we built every site in every language any site was in. At the moment this will build each site in every language that there is at least one file of that language in the site. So if a site has on file in lang AA, it will be built in lang AA. But if it has no files, it will not be built in that lang. This is a performance enhancement, will do benchmarks later. Some more complex heuristics for when we do/do not use a language for a file are being discussed in https://git.fsfe.org/FSFE/fsfe-website/issues/4601 .

delliott changed title from ~~WIP: feat: better delination of purpose of phases~~ to WIP: feat: build by site

2025-05-17 22:51:59 +00:00

delliott added the design translations labels 2025-05-17 22:55:31 +00:00

delliott requested review from tobiasd 2025-05-17 22:56:06 +00:00

delliott requested review from bonnie 2025-05-17 22:56:17 +00:00

delliott commented

2025-05-17 23:01:05 +00:00

Main difference I can see in builds on master vs this pr are

(Languages identified by 2 letter codes)

fsfe.org no longer built in bs, fa.
drm.info no longer built in ar, bg, bs, ca, cs, da, el, et, fi, hr
pdfreaders now build in eo, gl, he. (Were not before as those languages are not present in global/languages. That should be added)
pdfreaders now longer built in ja, mk, sk, sq, sr uk

Main difference I can see in builds on master vs this pr are (Languages identified by 2 letter codes) - fsfe.org no longer built in bs, fa. - drm.info no longer built in ar, bg, bs, ca, cs, da, el, et, fi, hr - pdfreaders now build in eo, gl, he. (Were not before as those languages are not present in global/languages. That should be added) - pdfreaders now longer built in ja, mk, sk, sq, sr uk

delliott commented

2025-05-19 20:31:35 +00:00

So, after fixing the global language thing:

fsfe.org no longer built in bs, fa.
drm.info no longer built in ar, bg, bs, ca, cs, da, el, et, fi, hr
pdfreaders now built in eo, gl, he.
pdfreaders now longer built in bg, bs, ca, da, ja, mk, sk, sq, sr, uk,

So, after fixing the global language thing: - fsfe.org no longer built in bs, fa. - drm.info no longer built in ar, bg, bs, ca, cs, da, el, et, fi, hr - pdfreaders now built in eo, gl, he. - pdfreaders now longer built in bg, bs, ca, da, ja, mk, sk, sq, sr, uk,

delliott commented

2025-05-19 21:00:04 +00:00

So, some quick benchmarks give that this pr is a whole 7 seconds faster, at 6 min 7 secs vs 6 min 14 secs.

What a speed boost. Anyway, we might pick up a few more if we decide to stop building in languages with no useful files or whatever. And this is logically cleaner, regardless of speed.

There are probably speed benefits to be picked up by running the three site builds concurrently, but that would make the logging so messy as to be unusable.

So, some quick benchmarks give that this pr is a whole 7 seconds faster, at 6 min 7 secs vs 6 min 14 secs. What a speed boost. Anyway, we might pick up a few more if we decide to stop building in languages with no useful files or whatever. And this is logically cleaner, regardless of speed. There are probably speed benefits to be picked up by running the three site builds concurrently, but that would make the logging so messy as to be unusable.

delliott force-pushed feat/by-site from 1f33a04b9d to 6ea62ef2d7

2025-05-19 21:00:20 +00:00

Compare

delliott added 1 commit 2025-05-19 21:11:41 +00:00

feat: add missing kurdish

continuous-integration/drone/pr Build is passing

Details

7da60ddf27

delliott commented

2025-05-19 21:18:45 +00:00

So, I consider this good to merge, but I want to make sure that the changes in build languages given in #5031 (comment) are acceptable.

So, I consider this good to merge, but I want to make sure that the changes in build languages given in https://git.fsfe.org/FSFE/fsfe-website/pulls/5031#issuecomment-41337 are acceptable.

tobiasd approved these changes 2025-05-20 10:49:27 +00:00

tobiasd left a comment

looking good to me :)

tobiasd commented

2025-05-20 10:51:24 +00:00

The languages that are not used for building the pages in are those that have some translations in other pages, but not for the selected one right?

delliott commented

2025-05-20 12:16:35 +00:00

Yes. It used to be the case that if fsfe.org. DRM.info or any other site had on file in a language, then all sites were built in that lang.

Eg if DRM.info has a Kurdish translation but no other site does, all sites were built in Kurdish anyway.

Now we build each site in the languages it has.

So DRM.info would be built in Kurdish, but no other sites would be.

Yes. It used to be the case that if fsfe.org. DRM.info or any other site had on file in a language, then all sites were built in that lang. Eg if DRM.info has a Kurdish translation but no other site does, all sites were built in Kurdish anyway. Now we build each site in the languages it has. So DRM.info would be built in Kurdish, but no other sites would be.

👍 1

delliott changed title from ~~WIP: feat: build by site~~ to feat: build by site

2025-05-20 20:15:11 +00:00

bonnie commented

2025-05-26 06:44:58 +00:00

Sounds good to me :)

sofiaritz approved these changes 2025-05-26 08:33:08 +00:00

sofiaritz left a comment

looks great :)

tobiasd merged commit 9c15f9400a into master

2025-05-26 12:18:50 +00:00

tobiasd deleted branch feat/by-site