The beat schedule is set to once per minute for each of the three
environment provisioning tasks.
Adding a beat schedule surfaced two problems that are addressed here
with the following changes:
- Commit the SQLALchemy session in order to release the environment
lock. Otherwise the change to the `claimed_until` field is not
persisted.
- Set `none_as_null` on the JSOB fields on the `Environment`. This
avoids problems with querying on Postgres JSON fields that are empty.
This also adds a small change to the development command for the Celery
worker. Multiple child processes were executing the beat jobs, which
lead to exceptions for environment locks and confusing log output. This
contrains the dev command to a single Celery worker.
AT-AT needs to be able to track which user tasks failed and why. To
accomplish this we:
- Enabled Celery's results backend, which logs task results to a data
store; a Postgres table, in our case.
(https://docs.celeryproject.org/en/latest/userguide/tasks.html#result-backends)
- Created tables to track the relationships between the relevant models
(Environment, EnvironmentRole) and their task failures.
- Added an `on_failure` hook that tasks can use. The hook will add
records to the job failure tables.
Now a resource like an `Environment` has access to it task failures
through the corresponding failure table.
Notes:
- It might prove useful to use a real foreign key to the Celery results
table eventually. I did not do it here because it requires that we
explicitly store the Celery results table schema as a migration and
add a model for it. In the current implementation, AT-AT can be
agnostic about where the results live.
- We store the task results indefinitely, so it is important to specify
tasks for which we do not care about the results (like `send_mail`)
via the `ignore_result` kwarg.
In the future, an `application_invitation1 will not refer to a `user` until
someone accepts the invitation; they'll only reference an
`application_role`. When a user is invited to an application, the
inviter can specify the environments the invitee should have access to.
For this to be possible, an `environment_role` should reference an
`application_role`, because no `user` entity will be known at that time.
In addition to updating all the models and domain methods necessary for
this change, this commit deletes unused code and tests that were
dependent on `environment_roles` having a `user_id` foreign key.
- parent relation will not include applications or environments marked
as deleted
- domain classes will exclude deleted objects from selections
- changed some test factories to use domain_word for resource names,
because they were using person names and it bugged me
