Turns out you can't map multiple K8s resources over the same directory.
The K8s secret for the INI file and the ConfigMap for the uWSGI config
both map into /opt/atat/atst in the container. This caused errors when
the container tried to launch. Instead, we need to specify the full file
path for every file we're mapping into that directory to avoid
conflicts.
Updates the K8s config to enable extended uWSGI JSON logging again. This
commit updates the name of the ConfigMap for the uWSGI config to avoid
confusion.
This commit is the first part of consuming secrets from the Azure Key Vault. This will set up the required services to consume Azure's RBAC controls in the cluster, an identity to read the secrets, and the tool (FlexVol) to mount the secrets.
We should not run a redundant testing workflow on merges to master or
staging.
This also includes a quick fix to configure the FLASK_ENV for the main
site.
Our content security policy in non-dev environments didn't allow uploading to azure blob storage. This adds a configurable blob storage base URL to allow regions to specify which storage endpoint they expect the upload request to use.
This value is set as the Access-Control-Allow-Origin header value for
the application. When using Azure CDN, the CDN will consume this header
when it populates its cache and use it on subsequent requests.
It would be possible to make this the same as the Flask SERVER_NAME
value. We explicitly set SERVER_NAME for Celery worker processes because
they need that information to contruct URLs outside of the request cycle
(Flask can infer the server name within a request cycle). I decided not
to rely on SERVER_NAME though because it has side effects:
- It determines what `url_for` uses as the host domain (which would be
fine).
- It makes it so that the Flask app can only server requests to that
domain (probably fine, but it felt like too big a side effect).
Additionally, SERVER_NAME does not include the scheme. For all of these
reasons I opted to make CDN_ORIGIN a separate config value.
Supplying this will prevent queue clashes between various ATAT sites
sharing the same Redis instance.
Note that the Celery documentation is currently wrong about the name for
configuring this:
https://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_default_queue
It specifies `CELERY_TASK_DEFAULT_QUEUE`, but
`CELERY_DEFAULT_QUEUE` is the value that Celery currently looks for.
This appears to be fixed in on an upcoming release:
https://github.com/celery/celery/issues/5575
This is worth keeping an eye on, since the configuration key could
change in the future.
This is not the certificate setup we will use in production. I'd like to
merge this configuration as a reference point because this is the
easiest way to handle manual LetsEncrypt verification within the
cluster.
This allows NGINX to serve static files over HTTP from the
".well-known/acme-challenge" directory, which is necessary for certbot
validation of domain ownership.
This generalizes the deploy step into a configurable CircleCI command.
The available parameters are:
- `namespace`: the K8s namespace to alter
- `tag`: the docker tag to apply to the image
The script for applying migrations to the K8s environment and the
corresponding K8s Job config have been generalized so that they can be
configured to run in the specified namespace.
The main workflow has been updated so that the appropriate deployment
will happen, depending on whether we are merging to staging or master.
In the future, we could look to add an additional workflow based around
Git tags for production.
Note that this also removes the creation of the `latest` tag from CD.
That tag is no longer hard-coded into our K8s config and so there's no
longer a need to update it in our container registry.
Adds a [kustomize](https://github.com/kubernetes-sigs/kustomize) overlay
for a new staging environment. Additionally, adds environment variables
in the place of certain pieces of information that need to be templated.
The K8s README ("deploy/README.md") has been updated to reflect the new
method for applying config.
This commit also removes the configuration for the AWS cluster and
references to AWS in the README.
This will allow us to force SSL connections to the database in
production by setting two values:
- PGSSLMODE should be set to "verify-full". This forces the client to
verify the server against a known CA: https://www.postgresql.org/docs/10/libpq-ssl.html
- PGSSLROOTCERT should be set to the path of the public cert for the
relevant CA.
When the database connection is made, these values are passed to the
adapter. For local development, PGSSLMODE is set to "prefer" and
PGSSLROOTCERT is left unset.
Kubernetes config has been added to maintain the root CAs for both Azure
and AWS as k8s ConfigMap objects. These are mounted into the containers
and referenced by PGSSLROOTCERT in the container environment.
Because I pushed the environment variable changes to the cluster
already, psycopg2 was automatically trying to connect to the database
using the file specified in PGSSLROOTCERT. That ConfigMap was not
mounted into the migrations container, so I'm doing that here.
The Celery worker cannot render URLs for the app without having a
SERVER_NAME value set. AT-AT's ability to send notifications when an
environment is ready is broken as a result.
This commit sets a null default value for SERVER_NAME in the default
config file. A setting must exist in the INI file in order to be
over-written by an environment variable, which is why we declare it as
null here. There is an additional kwarg, "allow_no_value", that must be
passed to ConfigParser to allow null values.
This also applies the correct domains as SERVER_NAME environment
variables in the Kubernetes ConfigMaps for the AWS and Azure Celery
workers.
With this configuration, all Kubernetes logs within the ATAT cluster
will be sent to AWS CloudWatch.
Note that this requires applying an additional IAM policy to the worker
nodes' role.
The Minikube version of the cluster has some differences from the main
config (noted in the README) but will be useful for for future DevOps
development.
We will run a separate pod for the beat worker. There should only ever
be a single beat worker (to avoid redundant work) so the number of
replicas needs to be managed independently.
This adds both the Kubernetes config for the new pod and additional
CircleCI config to swap a new image into the pod during CD.
This will allow Kubernetes resources that only pull images occasionally
(i.e., k8s jobs) to point to a static tag name, "latest", that is updated
regularly. It also means we can refer to that image in the k8s config
tracked in the repo, instead of out-of-date images.
Celery provides a more robust set of queueing options for both tasks and
worker processes. Updates include:
- infrastructure necessary to run Celery, including celery entrypoint
- backgrounded functions are now imported directly from atst.jobs
- update tests as-needed
- update kubernetes worker pod command
This applies configuration changes for the Flask app and adds changes to
the Dockerfile so that the build can make a CSP-specific JS bundle. It
adds `write_dotenv` script that creates the appropriate `.env` file for
the `parcel` bundler depending on how the `CSP` environment variable is
set.
- Configure K8s environment variables for Flask CSP usage
- Supply default CSP config setting to Flask app
- Declare the CSP arg in the Dockerfile
- Supply extra Docker build args to CD
- Fix top-level reference to boto3 in file_upload module
- Add back missing sample NGINX config for docker-compose build
Most build tools (i.e., the CircleCI Orbs) expect a Dockerfile in the
repo root. Rather than have to configure an exception everywhere, put it
where most people/tools expect it to be.
Dockerfile is now a single multi-stage build that relies on a Python 3.7
base image.
Notes:
- This builds uWSGI with a `pip install` because the Alpine vendored
uWSGI is built against Python 3.6.
- Adds a docker-compose file that can be used for testing that the build
works. It is not usable for development purposes because it creates a
static copy of the application.
