Because I pushed the environment variable changes to the cluster
already, psycopg2 was automatically trying to connect to the database
using the file specified in PGSSLROOTCERT. That ConfigMap was not
mounted into the migrations container, so I'm doing that here.
The Minikube version of the cluster has some differences from the main
config (noted in the README) but will be useful for for future DevOps
development.
Add a command to the test script to output up-to-date Vue component
templates. Most of the Vue component tests rely on HTML templates built
from Jinja.
The beat schedule is set to once per minute for each of the three
environment provisioning tasks.
Adding a beat schedule surfaced two problems that are addressed here
with the following changes:
- Commit the SQLALchemy session in order to release the environment
lock. Otherwise the change to the `claimed_until` field is not
persisted.
- Set `none_as_null` on the JSOB fields on the `Environment`. This
avoids problems with querying on Postgres JSON fields that are empty.
This also adds a small change to the development command for the Celery
worker. Multiple child processes were executing the beat jobs, which
lead to exceptions for environment locks and confusing log output. This
contrains the dev command to a single Celery worker.
The Kubernetes CronJob for syncing CRLs syncs them to a temporary folder
and then copies them to the real location once the sync is complete. If
the temporary folder is empty, the `cp` command throws an error. This
updates the bash script that manages the sync so that it will skip the
copy command if the temporary location is empty.
Celery provides a more robust set of queueing options for both tasks and
worker processes. Updates include:
- infrastructure necessary to run Celery, including celery entrypoint
- backgrounded functions are now imported directly from atst.jobs
- update tests as-needed
- update kubernetes worker pod command
`script/seed_sample.py` was creating portfolio users with no names
because it was calling `Users.get_or_create_by_dod_id` with a DOD ID as
its only argument. This updates it to pass the rest of the profile
information for the sample user.
This adds the following:
- A detect-secrets dependency and a related script
(`script/detect_secrets`) to find and alert developers to secrets
added to the code. By default, the script will search staged and new,
unstaged files. It can optionally search only staged files.
- A whitelist, `.secrets.baseline`, that tracks instances of secrets or
false positives already in the repo.
- Modifies `script/test` to detect secrets as part of the test suite.
- Updates to the README regarding the use of detect-secrets.
In `sh` on Alpine Linux in our Docker images, the newline characters in
the script were being interpreted as literals. This substitutes a
HEREDOC instead.
This applies configuration changes for the Flask app and adds changes to
the Dockerfile so that the build can make a CSP-specific JS bundle. It
adds `write_dotenv` script that creates the appropriate `.env` file for
the `parcel` bundler depending on how the `CSP` environment variable is
set.
- Configure K8s environment variables for Flask CSP usage
- Supply default CSP config setting to Flask app
- Declare the CSP arg in the Dockerfile
- Supply extra Docker build args to CD
- Fix top-level reference to boto3 in file_upload module
- Add back missing sample NGINX config for docker-compose build
- Fix some python formatting and import issues
- Fix dockerfile to include sync-crls script
- Adjust sync-crls script to use paths and CLI tools available in the
Docker container
This adds a previous version of the CRL sync functionality back to the
repo, with some small adjustments. We now grab the CRLs directly from
their DISA URLs.
The CRL sync is handled by a kubernetes cronjob that sync the files to a
persistent volume that is mounted into each Flask app container.
Add CircleCI config for both CSPs to:
- build the Docker image and push it to the registry
- run a short-lived k8s job to apply migrations and see data
- update the images for the Flask pods and rq worker pods
The submodule is a leftover from when this project was intended to work
as a series of microservices. It was meant to provide common
functionality to the builds for every microservice. That's no longer the
case, and the submodule is a pain-point both in on-boarding new
developers and running the Docker build.
In the future, an `application_invitation1 will not refer to a `user` until
someone accepts the invitation; they'll only reference an
`application_role`. When a user is invited to an application, the
inviter can specify the environments the invitee should have access to.
For this to be possible, an `environment_role` should reference an
`application_role`, because no `user` entity will be known at that time.
In addition to updating all the models and domain methods necessary for
this change, this commit deletes unused code and tests that were
dependent on `environment_roles` having a `user_id` foreign key.
