This generalizes the deploy step into a configurable CircleCI command.
The available parameters are:
- `namespace`: the K8s namespace to alter
- `tag`: the docker tag to apply to the image
The script for applying migrations to the K8s environment and the
corresponding K8s Job config have been generalized so that they can be
configured to run in the specified namespace.
The main workflow has been updated so that the appropriate deployment
will happen, depending on whether we are merging to staging or master.
In the future, we could look to add an additional workflow based around
Git tags for production.
Note that this also removes the creation of the `latest` tag from CD.
That tag is no longer hard-coded into our K8s config and so there's no
longer a need to update it in our container registry.
whitespace
The validator ListItemRequired() was only checking for None and an empty
string, not for strings that were multiple whitespace characters. This
fixes this issue by checking each item with regex to make sure it
contains non whitespace characters
The filter remove_empty_string() also was not checking for strings that
were multiple whitespace characters. This was also fixed by using regex
tomake sure that the string contains non whitespace characters, and also
clips any trailing whitespace.
Adds a [kustomize](https://github.com/kubernetes-sigs/kustomize) overlay
for a new staging environment. Additionally, adds environment variables
in the place of certain pieces of information that need to be templated.
The K8s README ("deploy/README.md") has been updated to reflect the new
method for applying config.
This commit also removes the configuration for the AWS cluster and
references to AWS in the README.
In local development, the app will fail to start if it does not find the
directory specified by CRL_STORAGE_CONTAINER. This adds a few lines to
safely create that directory on startup and corresponding tests.
For local development, we symlink the USWDS fonts from the npm installed
copy into our static directory. This causes problems for the Docker
build because it is not expecting to find a pre-existing "static/fonts"
directory. This forcibly removes any existing "static/fonts" directory
to fix the issue.
This ensures that the CronJon responsible for updating CRLs is using the
most up-to-date image. Previously, it used the "latest" tag. Since the
K8s Docker environment caches image tags, "latest" would not always be
up-to-date.
AT-AT needs to maintain a key-value CRL cache where each key is the DER
byte-string of the issuer and the value is a dictionary of the CRL file
path and expiration. This way when it checks a client certificate, it
can load the correct CRL by comparing the issuers. This is preferable to
loading all of the CRLs in-memory. However, it still requires that AT-AT
load and parse each CRL when the application boots. Because of the size
of the CRLs and their parsed, in-memory size, this leads to the
application spiking to use nearly 900MB of memory (resting usage is
around 50MB).
This change introduces a small function to ad-hoc parse the CRL and
obtain the information in the CRL we need: the issuer and the
expiration. It does this by reading the CRL byte-by-byte until it
reaches the ASN1 sequence that corresponds to the issuer, and then looks
ahead to find the nextUpdate field (i.e., the expiration date). The
CRLCache class uses this function to build its cache and JSON-serializes
the cache to disk. If another AT-AT application process finds the
serialized version, it will load that copy instead of rebuilding it. It
also entails a change to the function signature for the init method of
CRLCache: now it expects the CRL directory as its second argument,
instead of a list of locations.
The Python script invoked by `script/sync-crls` will rebuild the
location cache each time it's run. This means that when the Kubernetes
CronJob for CRLs runs, it will refresh the cache each time. When a new
application container boots, it will get the refreshed cache.
This also adds a nightly CircleCI job to sync the CRLs and test that the
ad-hoc parsing function returns the same result as a proper parsing
using the Python cryptography library. This provides extra insurance
that the function is returning correct results on real data.
- detect-secrets was previously bumped to 0.13 but the baseline file was
not updated.
- mypy objects to the way the Azure Subscription type was defined. Since
the module is encapsulated for dependency injection, we can either
declare it as a generic or remove the type hint. I did the latter,
since I don't know that we gain anything by the former.
