Feature: optional JSON endpoint for data export #62

New Issue

max.mehl · 2021-12-06T15:04:50Z

max.mehl commented

2021-12-06 15:04:50 +00:00

Problem

Right now, if a service or website (e.g. publiccode.eu or in future the fsfe.org website) wants to extract data from a forms application (e.g. pmpc-sign), this basically only works when this service is on the same server as forms, and reads forms' JSON file.

This is bad, since we do not want to couple services just because of this, or even cannot (in case of the FSFE website). Also, it is a security concern since this service could access all kinds of sensible data, e.g. signers of an open letter that did not want to appear publicly.

Possible solution

I propose an optional, configurable endpoint for forms. It would allow us to export certain data under certain conditions and make them available via a defined URL, ideally in a JSON format.

The fields that are exported should be customisable.

Example

Let's say I want to export a few things for the pmpc-sign application in forms:

total number of signatures
name, country and comment of every entry that has permissionPub = yes.

This could be available via https://forms.fsfe.org/api/pmpc-sign.json.

That can then be used in other services, e.g.:

pmpc-website where you can define remote JSON data as data source: https://gohugo.io/templates/data-templates/#get-remote-data
JavaScript to replace a placeholder on a website (e.g. fsfe.org) with the actual numbers and data from the JSON endpoint

Definition of fields and conditions

Probably, we need to settle on a basic language to define the exported fields and their conditions. IMHO this could be quite raw, like an SQL dialect, jq expressions or raw python, or something customised.

In any case I think we should make every field optional so that we don't expose unwanted data.

Blockers

This would actually remove the blocker for #25. But perhaps it's sane to do tackle #25 first and then immediately implement this feature so we don't have to tweak the same function twice.

## Problem Right now, if a service or website (e.g. publiccode.eu or in future the fsfe.org website) wants to extract data from a forms application (e.g. pmpc-sign), this basically only works when this service is on the same server as forms, and reads forms' JSON file. This is bad, since we do not want to couple services just because of this, or even cannot (in case of the FSFE website). Also, it is a security concern since this service could access all kinds of sensible data, e.g. signers of an open letter that did not want to appear publicly. ## Possible solution I propose an optional, configurable endpoint for forms. It would allow us to export certain data under certain conditions and make them available via a defined URL, ideally in a JSON format. The fields that are exported should be customisable. ## Example Let's say I want to export a few things for the `pmpc-sign` application in forms: * total number of signatures * `name`, `country` and `comment` of every entry that has `permissionPub = yes`. This could be available via `https://forms.fsfe.org/api/pmpc-sign.json`. That can then be used in other services, e.g.: * pmpc-website where you can define remote JSON data as data source: https://gohugo.io/templates/data-templates/#get-remote-data * JavaScript to replace a placeholder on a website (e.g. fsfe.org) with the actual numbers and data from the JSON endpoint ## Definition of fields and conditions Probably, we need to settle on a basic language to define the exported fields and their conditions. IMHO this could be quite raw, like an SQL dialect, `jq` expressions or raw python, or something customised. In any case I think we should make every field optional so that we don't expose unwanted data. ## Blockers This would actually remove the blocker for #25. But perhaps it's sane to do tackle #25 first and then immediately implement this feature so we don't have to tweak the same function twice.

florian.vuillemot was assigned by max.mehl

2022-02-11 13:42:44 +00:00

max.mehl added the

prio:mid

label 2022-02-11 13:44:09 +00:00

max.mehl commented

2022-09-08 12:17:33 +00:00

Poster

One very special idea for avoiding Javascript in this scenario:

JavaScript to replace a placeholder on a website (e.g. fsfe.org) with the actual numbers and data from the JSON endpoint

How about a dynamic CSS file via which we could insert text using the content field (example). Then, e.g. on fsfe.org, we could give an element an ID and append/prepend e.g. the number of signatures. One would have to test how thois behaves with caching but it's an idea 🤷

One very special idea for avoiding Javascript in this scenario: > JavaScript to replace a placeholder on a website (e.g. fsfe.org) with the actual numbers and data from the JSON endpoint How about a dynamic CSS file via which we could insert text using the `content` field ([example](https://css-tricks.com/almanac/selectors/a/after-and-before/)). Then, e.g. on fsfe.org, we could give an element an ID and append/prepend e.g. the number of signatures. One would have to test how thois behaves with caching but it's an idea :shrug:

linus added this to the 2023-1 milestone 2023-03-14 09:14:09 +00:00

Sign in to join this conversation.