AT-AT needs to be able to track which user tasks failed and why. To
accomplish this we:
- Enabled Celery's results backend, which logs task results to a data
store; a Postgres table, in our case.
(https://docs.celeryproject.org/en/latest/userguide/tasks.html#result-backends)
- Created tables to track the relationships between the relevant models
(Environment, EnvironmentRole) and their task failures.
- Added an `on_failure` hook that tasks can use. The hook will add
records to the job failure tables.
Now a resource like an `Environment` has access to it task failures
through the corresponding failure table.
Notes:
- It might prove useful to use a real foreign key to the Celery results
table eventually. I did not do it here because it requires that we
explicitly store the Celery results table schema as a migration and
add a model for it. In the current implementation, AT-AT can be
agnostic about where the results live.
- We store the task results indefinitely, so it is important to specify
tasks for which we do not care about the results (like `send_mail`)
via the `ignore_result` kwarg.
CLINS have a special ordering:
- First, they are sorted by the last three digits
- Then, they are sorted by the first digit
Trying to add CLIN sorting logic to the relationship field in the task
order proved to be more challenging than expected. So, a separate
property was defined in order to access the clins in sorted order.
Debug mode allows route integration tests to raise explicit exceptions on
errors, instead of returning error pages. Some portions of the test
suite need to be able to ignore exceptions (the response is not under
test) so they use a separate pytest fixture version of the app and
client that are configured with debug disabled, as it would be in
production.
Celery provides a more robust set of queueing options for both tasks and
worker processes. Updates include:
- infrastructure necessary to run Celery, including celery entrypoint
- backgrounded functions are now imported directly from atst.jobs
- update tests as-needed
- update kubernetes worker pod command
There are still a few tests that rely on physical certificate files.
These are slightly more complicated to replace with pytest fixtures.
Updated the script for generating a CRL chain so that the CRL is good
for roughly ten years and regenerated the certs and CRL.
This updates the `Portfolios.for_user` method to screen out deleted
ApplicationRole entities. For extra assurance, we also mark application
roles as disabled when they are deleted.
- Applies our previous CSP namespacing pattern to the upload classes.
- Removes code and config for previous uploader implementation.
- Removes Attachment model's ability to upload files directly and
adjusts tests that expected that behavior.
