MP Scrape UI

This is a Django-based UI for mp-scrape.

Development

• Install Redis
• Install dependencies: pip install Django Celery django-celery-beat redis mp-scrape-core mp-scrape-cli mp-scrape-export-sqlite mp-scrape-source-bundestag mp-scrape-source-euparl mp-scrape-source-nrw
• Run python3 manage.py migrate
• Place your workflows
• Run python3 manage.py start_workflows
• Run python3 manage.py runserver
• Run python3 -m celery -A mp_scrape_ui worker -l DEBUG

Workflows

Workflows are stored in the mp_scrape_ui/workflows folder. You can find example workflows at the docs/workflows/examples folder. You can read more about the specifics of mp-scrape-ui workflows in docs/workflows/README.md.

Authentication

You need to create an administrative account in order to accept access requests. In order to do this, run python3 manage.py createsuperuser and the instructions.

There are three tiers of users:

1. unapproved Those who have not been approved by an administrator. They do not have access to anything.
2. staff They have been approved by an administrator. They have access to the scraped databases.
3. administrator They have been manually promoted by an administrator. They have access to the scraped databases, administration interface, task status, and direct database access through /admin.

Promote new user to staff

1. (User) Go to /accounts/signup and create an account
2. (User) Log into your newly created account
3. (User) Share your Username and User ID with an administrator.
4. (Admin) Go to /accounts/admin, look for the user with matching Username and ID
5. (Admin) Press Make Staff to allow access to the user.

Demote staff to unapproved user

Note

This can only be performed by an administrator.

1. Go to /accounts/amdin and look for the user that you want to demote.
2. Press Remove Staff to disallow access to the user.

(Optional)

1. Go to /admin.
2. In the Authentication and Authorization section, select Users.
3. Look for the user that you want to permanently delete and click the checkbox on the left.
4. Press the dropdown nearby Action: and look for Delete selected users.
5. Press Go.
6. Review the summary and press Yes, I'm sure.
7. Make sure that the page says at the top Successfully deleted 1 user.. If the result is different take a screenshot of the page, copy the error message, write down your exact actions that lead up to this moment, and forward this information to the instance's maintainer.

Promoting staff to administrator

Note

This can only be performed by an administrator.

1. Go to /admin
2. In the Authentication and Authorization section, select Users.
3. Look for the user that you want to permanently delete and click their username.
4. Scroll down, look for the Superuser status checkbox, and check it.
5. Scroll down and press SAVE
6. Make sure that the page says at the top The user “<username>” was changed successfully.. If the result is different take a screenshot of the page, copy the error message, write down your exact actions that lead up to this moment, and forward this information to the instance's maintainer.