dandds
GitHub
commit
cf6bd7dcd0
4
.gitignore
vendored
4
.gitignore
vendored
@ -58,3 +58,7 @@ browserstacklocal
|
||||
|
||||
# python config
|
||||
.python-version
|
||||
|
||||
# configuration files
|
||||
override.ini
|
||||
atst-overrides.ini
|
||||
|
||||
200
deploy/README.md
Normal file
200
deploy/README.md
Normal file
@ -0,0 +1,200 @@
|
||||
# Kubernetes Deployment Configuration
|
||||
|
||||
This folder contains Kubernetes deployment configuration for AWS and Azure. The following assumes that you have `kubectl` installed and configured with permissions to a cluster in one of the two CSPs.
|
||||
|
||||
## Applying K8s configuration
|
||||
|
||||
Note that the images specified in the config are out-of-date. CI/CD updates them automatically within the clusters. Be careful when applying new config that you don't rotate the image to an older copy that is out-of-date.
|
||||
|
||||
Depending on how your `kubectl` config is set up, these commands may need to be adjusted. If you have configuration for both clusters, you may need to specify the `kubectl` context for each command with the `--context` flag (something like `kubectl --context=aws [etc.]` or `kubectl --context=azure [etc.]`).
|
||||
|
||||
### Apply the config to an AWS cluster
|
||||
|
||||
Applying the AWS configuration requires that you have an Elastic File Storage (EFS) volume available to your EC2 node instances. The EFS ID should be set as an environment variable before you apply the AWS config:
|
||||
|
||||
```
|
||||
export EFSID=my-efs-volume-id
|
||||
```
|
||||
|
||||
First apply all the config:
|
||||
|
||||
```
|
||||
kubectl apply -f deploy/aws/
|
||||
```
|
||||
|
||||
Then apply the storage class config using `envsubst`:
|
||||
|
||||
```
|
||||
envsubst < deploy/aws/storage-class.yml | kubectl apply -f -
|
||||
```
|
||||
|
||||
When applying configuration changes, be careful not to over-write the storage class configuration without the environment variable substituted.
|
||||
|
||||
### Apply the config to an Azure cluster
|
||||
|
||||
To apply the configuration to a new cluster, run:
|
||||
|
||||
```
|
||||
kubectl apply -f deploy/azure/
|
||||
```
|
||||
|
||||
## Secrets and Configuration
|
||||
|
||||
### atst-overrides.ini
|
||||
|
||||
Production configuration values are provided to the ATAT Flask app by writing an `atst-overrides.ini` file to the running Docker container. This file is stored as a Kubernetes secret. It contains configuration information for the database connection, mailer, etc.
|
||||
|
||||
To update the configuration, you can do the following:
|
||||
|
||||
```
|
||||
kubectl -n atat get secret atst-config-ini -o=jsonpath='{.data.override\.ini}' | base64 --decode > override.ini
|
||||
```
|
||||
|
||||
This base64 decodes the secret and writes it to a local file called `override.ini`. Make any necessary config changes to that file.
|
||||
|
||||
To apply the new config, first delete the existing copy of the secret:
|
||||
|
||||
```
|
||||
kubectl -n atat delete secret atst-config-ini
|
||||
```
|
||||
|
||||
Then create a new copy of the secret from your updated copy:
|
||||
|
||||
```
|
||||
kubectl -n atat create secret generic atst-config-ini --from-file=./override.ini
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Be careful not to check the override.ini file into source control.
|
||||
- Be careful not to overwrite one CSP cluster's config with the other's. This will break everything.
|
||||
|
||||
### nginx-client-ca-bundle
|
||||
|
||||
(NOTE: This really doesn't need to be a secret since these are public certs. A good change would be to convert it to a k8s configmap.)
|
||||
|
||||
This is the PEM certificate file of the DoD Certificate Authority certs. This must be available for CAC authentication.
|
||||
|
||||
A local copy of the certs are stored in the repo at `ssl/client-certs/ca-chain.pem`. It can be updated by running `script/sync-dod-certs`. When creating a new cluster, you can copy the cert file to the repo root:
|
||||
|
||||
```
|
||||
cp ssl/client-certs/ca-chain.pem client-ca-bundle.pem
|
||||
```
|
||||
|
||||
and then create a new secret from it:
|
||||
|
||||
```
|
||||
kubectl -n atat create secret generic nginx-client-ca-bundle --from-file=./client-ca-bundle.pem
|
||||
```
|
||||
|
||||
### nginx-htpasswd
|
||||
|
||||
If the site is running in dev mode, the `/login-dev` endpoint is available. This endpoint is protected by basic HTTP auth. To create a new password file, run:
|
||||
|
||||
```
|
||||
htpasswd -c ./htpasswd atat
|
||||
```
|
||||
|
||||
Enter a new password string when prompted. Then create the secret:
|
||||
|
||||
```
|
||||
kubectl -n atat create secret generic nginx-htpasswd --from-file=./htpasswd
|
||||
```
|
||||
|
||||
## SSL/TLS
|
||||
|
||||
(NOTE: We should get cert-manager working for automatic updates of the certificates.)
|
||||
|
||||
The NGINX instance in the ATAT pod handles SSL/TLS termination for both the main domain and authentication domain. The certificates are stored as a k8s TLS secret. Currently, certs are obtained from Let's Encrypt by running certbot in manual mode. This section will walk through that process.
|
||||
|
||||
For context, you should be familiar with [ACME HTTP-01 challenge](https://letsencrypt.org/docs/challenge-types/) method for proving ownership of a domain.
|
||||
|
||||
A rough example of the necessary config changes can be found at this commit: [`d81b8a03b2bb407e04cd4265fa90f4cf7ab82b9e`](https://github.com/dod-ccpo/atst/commit/d81b8a03b2bb407e04cd4265fa90f4cf7ab82b9e)
|
||||
|
||||
To proceed, you will need to install [certbot](https://certbot.eff.org/docs/install.html). If you are on macOs and have `homebrew` installed, `brew install certbot`. These instructions assume a baseline familiarity with NGINX config, Kubernetes, and `kubectl`.
|
||||
|
||||
As a broad overview, we will:
|
||||
|
||||
- Ask certbot for certificates for the two domains
|
||||
- Get ACME challenge data from certbot
|
||||
- Make that data available to the NGINX container in the ATAT pod in our cluster
|
||||
- Update the NGINX config so that it can server the ACME challenge
|
||||
|
||||
Once this is done, certbot will be able to confirm that we own the domains and will issue certificates. Then we can make those certs available as TLS secrets in the cluster.
|
||||
|
||||
These steps should work for updating an existing site. If you are setting up HTTPS for a new site, make sure DNS is assigned for your two domains.
|
||||
|
||||
### Start certbot
|
||||
|
||||
First start certbot in manual mode:
|
||||
|
||||
```
|
||||
certbot --manual
|
||||
```
|
||||
|
||||
(You may need to supply `--config-dir`, `--work-dir`, and `--logs-dir` options; certbot may try to write to directories that require root privileges by default.)
|
||||
|
||||
You will be prompted to enter the domain names you want a cert for. Enter **both** the main and auth domains. For instance:
|
||||
|
||||
```
|
||||
jedi.atat.code.mil,jedi-auth.atat.code.mil
|
||||
```
|
||||
|
||||
You must agree to have your IP logged to proceed.
|
||||
|
||||
### The ACME challenge files
|
||||
|
||||
First you will be presented with an ACME challenge for the main domain. Create a file with the challenge. The file should be named for the last part of the URL path and contain the challenge data.
|
||||
|
||||
Create a k8s secret from the challenge file:
|
||||
|
||||
```
|
||||
kubectl -n atat create secret generic acme --from-file=./dpYfQi4C2qgH1WW_XZmFPbahqOsXJKh64ZOqJCWB4q0
|
||||
```
|
||||
|
||||
(Substitute your challenge file name for the `dpY` string in the example.)
|
||||
|
||||
Refer to the sample commit mentioned above for examples of Kubernetes config necessary to make the secret available as a file to the NGINX container. Once the YAML has been updated, apply it to the cluster.
|
||||
|
||||
You will repeat this process for the auth domain. Note that the secret name for the auth domain should be `acme-auth`.
|
||||
|
||||
### NGINX config
|
||||
|
||||
Next, apply NGINX config to allow NGINX to server the ACME challenges. The NGINX config can be found in `deploy/{aws,azure}/atst-nginx-configmap.yml`.
|
||||
|
||||
There are two server blocks at the beginning of the config that redirect HTTP requests to the domains to their HTTPS equivalents. Temporarily alter these blocks so that they serve the ACME challenge:
|
||||
|
||||
```
|
||||
server {
|
||||
listen 8342;
|
||||
server_name jedi.atat.code.mil;
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
```
|
||||
|
||||
becomes:
|
||||
|
||||
```
|
||||
server {
|
||||
listen 8342;
|
||||
server_name jedi.atat.code.mil;
|
||||
root /usr/share/nginx/html;
|
||||
location /.well-known/acme-challenge/ {
|
||||
try_files $uri =404;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Do this for both the 8342 and 8342 blocks. Apply the config changes to the cluster. (You may have to rebuild the pods, since they will not inherit the updated NGINX config automatically.)
|
||||
|
||||
You can confirm that the cluster is serving the ACME challenges successfully by hitting the URLs certbot lists for the challenges. You should get your challenge file back when hitting the URL. You can hit [enter] in the certbot prompt, and it should issue certificates to a location on your machine.
|
||||
|
||||
### Create the TLS secret
|
||||
|
||||
Once you have obtained the certs, you can create the new TLS secret in the cluster. First delete the existing secret. (It should be named something like `csp-atat-code-mil-tls`, where `csp` is the name of the CSP.)
|
||||
|
||||
Then:
|
||||
|
||||
```
|
||||
kubectl -n atat create secret tls csp-atat-code-mil-tls --key="[path to the private key]" --cert="[path to the full chain]"
|
||||
```
|
||||
@ -1,24 +0,0 @@
|
||||
server {
|
||||
listen 80;
|
||||
server_name localhost;
|
||||
|
||||
#charset koi8-r;
|
||||
#access_log /var/log/nginx/host.access.log main;
|
||||
|
||||
location / {
|
||||
try_files $uri @app;
|
||||
}
|
||||
|
||||
location @app {
|
||||
include uwsgi_params;
|
||||
uwsgi_pass unix:///var/run/uwsgi/uwsgi.socket;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_VERIFY $ssl_client_verify;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_CERT $ssl_client_raw_cert;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_S_DN $ssl_client_s_dn;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_S_DN_LEGACY $ssl_client_s_dn_legacy;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_I_DN $ssl_client_i_dn;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_I_DN_LEGACY $ssl_client_i_dn_legacy;
|
||||
uwsgi_param HTTP_X_REQUEST_ID $request_id;
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,81 +0,0 @@
|
||||
# Adding a New Environment
|
||||
|
||||
## New Config
|
||||
|
||||
Add a new subfolder to this directory. You can copy `uat` or `tests`. You'll need to change any references to the previous environment name (i.e., `uat`) to your new environment name. This includes things like the k8s namespace and the subdomain for the new site.
|
||||
|
||||
## New Load Balancers
|
||||
|
||||
You need two new load balancers. Currently, these are managed in Rackspace. You will need one for the regular site and one for the auth domain. They should have all of the Kubernetes worker nodes attached. When attached to the LB for the main site domain, the nodes should point to port 32761, which is the port where all our NGINX ingress is managed. The auth LB nodes should point to a new port in the 32700 - 32799 range. The `nodePort` specified in your new environment's config should match this port.
|
||||
|
||||
## Initially Apply the New Config
|
||||
|
||||
Apply your new environment config to create the namespace. Pod creation will fail at this point.
|
||||
|
||||
```
|
||||
kubectl -n my-new-env apply -f deploy/kubernetes/my-new-env/
|
||||
```
|
||||
|
||||
## Create New Secrets
|
||||
|
||||
You should then copy and duplicate the secrets from another environment.
|
||||
|
||||
### Adjust the INI Config for the New Site
|
||||
|
||||
You can duplicate the override.ini file from one of the existing sites. For instance, to get the `atst-config-ini` secret for the UAT env:
|
||||
|
||||
```
|
||||
kubectl -n atat-uat get secret atst-config-ini -o yaml > uat-secrets.yml
|
||||
```
|
||||
|
||||
You can copy the base64 secret content from the `uat-secrets.yml` file. Decode it into a new `override.ini` file:
|
||||
|
||||
```
|
||||
echo '[Paste in the long base64 string here]' | base64 --decode > override.ini
|
||||
```
|
||||
|
||||
Edit and adjust the application config as needed for your new site. Then add it as a secret:
|
||||
|
||||
```
|
||||
kubectl -n my-new-env create secret generic atst-config-ini --from-file=override.ini
|
||||
```
|
||||
|
||||
### Add a New htpasswd
|
||||
|
||||
Create a new htpasswd to protect the dev login of the new site:
|
||||
|
||||
```
|
||||
htpasswd -c htpasswd atat
|
||||
```
|
||||
|
||||
You'll be prompted for the new password. Then add it as a secret:
|
||||
|
||||
```
|
||||
kubectl -n my-new-env create secret generic atst-nginx-htpasswd --from-file=htpasswd
|
||||
```
|
||||
|
||||
### Duplicate the Rest
|
||||
|
||||
You should also the `dhparam-4096` and `nginx-client-ca-bundle` secrets. These can be copied from an existing environment to yours without any changes. The TLS secrets and token will be handled by another service.
|
||||
|
||||
## Disable SSL
|
||||
|
||||
In order for [kube-lego](https://github.com/jetstack/kube-lego) to generate new certs for your site, you have to temporarily disable SSL for your new load balancers.
|
||||
|
||||
For both your LBs, set them to use HTTP/80. Delete the attached nodes for both and re-add them, setting them to use port 32760. This will allow kube-lego to do its job.
|
||||
|
||||
To monitor the process, find the pod ID for the kube-lego worker. To find it, look in the output for:
|
||||
|
||||
```
|
||||
kubectl -n kube-system get all
|
||||
```
|
||||
|
||||
Then once you know the pod name:
|
||||
|
||||
```
|
||||
kubectl -n kube-system --kubeconfig ~/.kube/atat log kube-lego-b96b7bc5c-9fmcv -f --tail=10
|
||||
```
|
||||
|
||||
You will see output about kube-lego attempts to create certs for your new site.
|
||||
|
||||
Once kube-lego is successful, you should restore your load balancers to the config they had initially. Additionally, you should enable HTTPS redirects.
|
||||
@ -1,38 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
|
||||
set -o pipefail
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
|
||||
# Decode and save the K8S CA cert
|
||||
echo "${K8S_CA_CRT}" | base64 -d - > "${HOME}/k8s_ca.crt"
|
||||
|
||||
# Setup the local kubectl client
|
||||
kubectl config set-context atst-deployer \
|
||||
--cluster=atat-cluster \
|
||||
--user=atat-deployer \
|
||||
--namespace=atat
|
||||
|
||||
kubectl config set-cluster atat-cluster \
|
||||
--embed-certs=true \
|
||||
--server="${K8S_ENDPOINT}" \
|
||||
--certificate-authority="${HOME}/k8s_ca.crt"
|
||||
|
||||
kubectl config set-credentials atat-deployer --token="$(echo ${K8S_USER_TOKEN} | base64 -d -)"
|
||||
|
||||
kubectl config use-context atst-deployer
|
||||
kubectl config current-context
|
||||
|
||||
echo
|
||||
echo "Current ATST Deployment Details:"
|
||||
kubectl -n atat get deployment.apps/atst -o yaml
|
||||
|
||||
# Remove the K8S CA file when the script exits
|
||||
function cleanup {
|
||||
printf "Cleaning up...\n"
|
||||
rm -vf "${HOME}/k8s_ca.crt"
|
||||
printf "Cleaning done."
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
@ -1,126 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
app: atst-debugger
|
||||
name: atst-debugger-v1
|
||||
namespace: atat
|
||||
spec:
|
||||
securityContext:
|
||||
fsGroup: 101
|
||||
containers:
|
||||
- name: atst
|
||||
image: registry.atat.code.mil:443/atst-prod:6329f8e
|
||||
args: ["/bin/bash", "-c", "while true; do date; sleep 45; done"]
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: atst-envvars
|
||||
volumeMounts:
|
||||
- name: atst-config
|
||||
mountPath: "/opt/atat/atst/atst-overrides.ini"
|
||||
subPath: atst-overrides.ini
|
||||
- name: nginx-client-ca-bundle
|
||||
mountPath: "/opt/atat/atst/ssl/server-certs/ca-chain.pem"
|
||||
subPath: client-ca-bundle.pem
|
||||
- name: uwsgi-config
|
||||
mountPath: "/opt/atat/atst/uwsgi-config.ini"
|
||||
subPath: uwsgi-config.ini
|
||||
- name: uwsgi-socket-dir
|
||||
mountPath: "/var/run/uwsgi"
|
||||
- name: atst-nginx
|
||||
image: nginx:alpine
|
||||
ports:
|
||||
- containerPort: 8442
|
||||
name: http
|
||||
- containerPort: 8443
|
||||
name: https
|
||||
volumeMounts:
|
||||
- name: nginx-auth-tls
|
||||
mountPath: "/etc/ssl/private"
|
||||
- name: nginx-client-ca-bundle
|
||||
mountPath: "/etc/ssl/client-ca-bundle.pem"
|
||||
subPath: client-ca-bundle.pem
|
||||
- name: nginx-config
|
||||
mountPath: "/etc/nginx/conf.d/atst.conf"
|
||||
subPath: atst.conf
|
||||
- name: nginx-dhparam
|
||||
mountPath: "/etc/ssl/dhparam.pem"
|
||||
subPath: dhparam.pem
|
||||
- name: nginx-htpasswd
|
||||
mountPath: "/etc/nginx/.htpasswd"
|
||||
subPath: .htpasswd
|
||||
- name: uwsgi-socket-dir
|
||||
mountPath: "/var/run/uwsgi"
|
||||
imagePullSecrets:
|
||||
- name: regcred
|
||||
volumes:
|
||||
- name: atst-config
|
||||
secret:
|
||||
secretName: atst-config-ini
|
||||
items:
|
||||
- key: override.ini
|
||||
path: atst-overrides.ini
|
||||
mode: 0644
|
||||
- name: nginx-auth-tls
|
||||
secret:
|
||||
secretName: auth-atst-ingress-tls
|
||||
items:
|
||||
- key: tls.crt
|
||||
path: auth.atat.crt
|
||||
mode: 0644
|
||||
- key: tls.key
|
||||
path: auth.atat.key
|
||||
mode: 0640
|
||||
- name: nginx-client-ca-bundle
|
||||
secret:
|
||||
secretName: nginx-client-ca-bundle
|
||||
items:
|
||||
- key: client-ca-bundle.pem
|
||||
path: client-ca-bundle.pem
|
||||
mode: 0666
|
||||
- name: nginx-config
|
||||
configMap:
|
||||
name: atst-nginx
|
||||
items:
|
||||
- key: nginx-config
|
||||
path: atst.conf
|
||||
- name: nginx-dhparam
|
||||
secret:
|
||||
secretName: dhparam-4096
|
||||
items:
|
||||
- key: dhparam.pem
|
||||
path: dhparam.pem
|
||||
mode: 0640
|
||||
- name: nginx-htpasswd
|
||||
secret:
|
||||
secretName: atst-nginx-htpasswd
|
||||
items:
|
||||
- key: htpasswd
|
||||
path: .htpasswd
|
||||
mode: 0640
|
||||
- name: uwsgi-config
|
||||
configMap:
|
||||
name: atst-config
|
||||
items:
|
||||
- key: uwsgi-config
|
||||
path: uwsgi-config.ini
|
||||
mode: 0644
|
||||
- name: uwsgi-socket-dir
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
restartPolicy: Never
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels:
|
||||
app: atst-debugger
|
||||
name: atst-debugger
|
||||
namespace: atat
|
||||
spec:
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: 8442
|
||||
selector:
|
||||
app: atst-debugger
|
||||
@ -1,41 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: atat-deployer
|
||||
namespace: atat
|
||||
---
|
||||
kind: Role
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
namespace: atat
|
||||
name: atat-deploy-role
|
||||
rules:
|
||||
- apiGroups: ["extensions", "apps"]
|
||||
resources: ["deployments"]
|
||||
verbs: ["get", "watch", "create", "update", "patch"]
|
||||
- apiGroups: [""]
|
||||
resources: ["services"]
|
||||
verbs: ["get", "create", "update", "patch"]
|
||||
- apiGroups: [""]
|
||||
resources: ["configmaps"]
|
||||
verbs: ["get", "create", "update", "patch"]
|
||||
- apiGroups: ["extensions", "apps"]
|
||||
resources: ["ingresses"]
|
||||
verbs: ["get", "create", "update", "patch"]
|
||||
---
|
||||
kind: RoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: atst-role-binding
|
||||
namespace: atat
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: atat-deployer
|
||||
namespace: atat
|
||||
- kind: ServiceAccount
|
||||
name: atat-deployer
|
||||
namespace: atat-test
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: atat-deploy-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
@ -1,13 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: atst-envvars
|
||||
namespace: atat
|
||||
data:
|
||||
TZ: UTC
|
||||
FLASK_ENV: dev
|
||||
OVERRIDE_CONFIG_FULLPATH: /opt/atat/atst/atst-overrides.ini
|
||||
UWSGI_CONFIG_FULLPATH: /opt/atat/atst/uwsgi-config.ini
|
||||
CRL_STORAGE_PROVIDER: CLOUDFILES
|
||||
LOG_JSON: "true"
|
||||
@ -1,99 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: atst-nginx
|
||||
namespace: atat
|
||||
data:
|
||||
nginx-config: |-
|
||||
server {
|
||||
server_name www.atat.code.mil atat.code.mil;
|
||||
access_log /var/log/nginx/access.log json;
|
||||
listen 8442;
|
||||
listen [::]:8442 ipv6only=on;
|
||||
if ($http_x_forwarded_proto != 'https') {
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
location /login-redirect {
|
||||
return 301 https://auth.atat.code.mil$request_uri;
|
||||
}
|
||||
location /login-dev {
|
||||
try_files $uri @appbasicauth;
|
||||
}
|
||||
location / {
|
||||
try_files $uri @app;
|
||||
}
|
||||
location @app {
|
||||
include uwsgi_params;
|
||||
uwsgi_pass unix:///var/run/uwsgi/uwsgi.socket;
|
||||
uwsgi_param HTTP_X_REQUEST_ID $request_id;
|
||||
}
|
||||
location @appbasicauth {
|
||||
include uwsgi_params;
|
||||
uwsgi_pass unix:///var/run/uwsgi/uwsgi.socket;
|
||||
auth_basic "Developer Access";
|
||||
auth_basic_user_file /etc/nginx/.htpasswd;
|
||||
uwsgi_param HTTP_X_REQUEST_ID $request_id;
|
||||
}
|
||||
}
|
||||
server {
|
||||
access_log /var/log/nginx/access.log json;
|
||||
server_name auth.atat.code.mil;
|
||||
listen 8443 ssl;
|
||||
listen [::]:8443 ssl ipv6only=on;
|
||||
# SSL server certificate and private key
|
||||
ssl_certificate /etc/ssl/private/auth.atat.crt;
|
||||
ssl_certificate_key /etc/ssl/private/auth.atat.key;
|
||||
# Set SSL protocols, ciphers, and related options
|
||||
ssl_protocols TLSv1.3 TLSv1.2;
|
||||
ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
|
||||
ssl_prefer_server_ciphers on;
|
||||
ssl_ecdh_curve secp384r1;
|
||||
ssl_dhparam /etc/ssl/dhparam.pem;
|
||||
# SSL session options
|
||||
ssl_session_timeout 4h;
|
||||
ssl_session_cache shared:SSL:10m; # 1mb = ~4000 sessions
|
||||
ssl_session_tickets off;
|
||||
# OCSP Stapling
|
||||
ssl_stapling on;
|
||||
ssl_stapling_verify on;
|
||||
resolver 8.8.8.8 8.8.4.4;
|
||||
# Request and validate client certificate
|
||||
ssl_verify_client on;
|
||||
ssl_verify_depth 10;
|
||||
ssl_client_certificate /etc/ssl/client-ca-bundle.pem;
|
||||
# Guard against HTTPS -> HTTP downgrade
|
||||
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; always";
|
||||
location / {
|
||||
return 301 https://www.atat.code.mil$request_uri;
|
||||
}
|
||||
location /login-redirect {
|
||||
try_files $uri @app;
|
||||
}
|
||||
location @app {
|
||||
include uwsgi_params;
|
||||
uwsgi_pass unix:///var/run/uwsgi/uwsgi.socket;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_VERIFY $ssl_client_verify;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_CERT $ssl_client_raw_cert;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_S_DN $ssl_client_s_dn;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_S_DN_LEGACY $ssl_client_s_dn_legacy;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_I_DN $ssl_client_i_dn;
|
||||
uwsgi_param HTTP_X_SSL_CLIENT_I_DN_LEGACY $ssl_client_i_dn_legacy;
|
||||
uwsgi_param HTTP_X_REQUEST_ID $request_id;
|
||||
}
|
||||
}
|
||||
nginx-json-log-config: |-
|
||||
log_format json escape=json
|
||||
'{'
|
||||
'"timestamp":"$time_iso8601",'
|
||||
'"msec":"$msec",'
|
||||
'"request_id":"$request_id",'
|
||||
'"remote_addr":"$remote_addr",'
|
||||
'"remote_user":"$remote_user",'
|
||||
'"request":"$request",'
|
||||
'"status":$status,'
|
||||
'"body_bytes_sent":$body_bytes_sent,'
|
||||
'"referer":"$http_referer",'
|
||||
'"user_agent":"$http_user_agent",'
|
||||
'"http_x_forwarded_for":"$http_x_forwarded_for"'
|
||||
'}';
|
||||
@ -1,54 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# deploy/kubernetes/atst-update-deploy.sh: Resets the sample data on the target
|
||||
# environment.
|
||||
|
||||
set -o pipefail
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
# set -o xtrace
|
||||
|
||||
# Config
|
||||
MAX_DEPLOY_WAIT='300'
|
||||
|
||||
if [[ $# -eq 0 ]]; then
|
||||
NAMESPACE=atat
|
||||
else
|
||||
NAMESPACE=$1
|
||||
fi
|
||||
|
||||
# Remove the K8S CA file when the script exits
|
||||
function cleanup {
|
||||
printf "Cleaning up...\n"
|
||||
rm -vf "${HOME}/k8s_ca.crt"
|
||||
printf "Cleaning done."
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# Decode and save the K8S CA cert
|
||||
echo "${K8S_CA_CRT}" | base64 -d - > "${HOME}/k8s_ca.crt"
|
||||
|
||||
# Setup the local kubectl client
|
||||
kubectl config set-context atst-deployer \
|
||||
--cluster=atat-cluster \
|
||||
--user=atat-deployer \
|
||||
--namespace=${NAMESPACE}
|
||||
|
||||
kubectl config set-cluster atat-cluster \
|
||||
--embed-certs=true \
|
||||
--server="${K8S_ENDPOINT}" \
|
||||
--certificate-authority="${HOME}/k8s_ca.crt"
|
||||
|
||||
kubectl config set-credentials atat-deployer --token="$(echo ${K8S_USER_TOKEN} | base64 -d -)"
|
||||
|
||||
kubectl config use-context atst-deployer
|
||||
kubectl config current-context
|
||||
|
||||
# we only need to run these commands against one existing pod
|
||||
ATST_POD=$(kubectl -n ${NAMESPACE} get pods -l app=atst -o custom-columns=NAME:.metadata.name --no-headers | sed -n 1p)
|
||||
# echo "kubectl -n ${NAMESPACE} exec ${ATST_POD} -- pipenv run python script/remove_sample_data.py"
|
||||
echo "removing sample data on pod ${ATST_POD}"
|
||||
kubectl -n ${NAMESPACE} exec ${ATST_POD} -- pipenv run python script/remove_sample_data.py
|
||||
echo "seeding sample data on pod ${ATST_POD}"
|
||||
kubectl -n ${NAMESPACE} exec ${ATST_POD} -- pipenv run python script/seed_sample.py
|
||||
|
||||
@ -1,64 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# deploy/kubernetes/atst-update-deploy.sh: Updates the existing ATST deployment
|
||||
# with a new source image
|
||||
|
||||
set -o pipefail
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
# set -o xtrace
|
||||
|
||||
# Config
|
||||
MAX_DEPLOY_WAIT='300'
|
||||
|
||||
if [[ $# -eq 0 ]]; then
|
||||
NAMESPACE=atat
|
||||
else
|
||||
NAMESPACE=$1
|
||||
fi
|
||||
|
||||
if [ "${IMAGE_NAME}x" = "x" ]
|
||||
then
|
||||
IMAGE_NAME="${ATAT_DOCKER_REGISTRY_URL}/${PROD_IMAGE_NAME}:${GIT_SHA}"
|
||||
fi
|
||||
|
||||
# Remove the K8S CA file when the script exits
|
||||
function cleanup {
|
||||
printf "Cleaning up...\n"
|
||||
rm -vf "${HOME}/k8s_ca.crt"
|
||||
printf "Cleaning done."
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# Decode and save the K8S CA cert
|
||||
echo "${K8S_CA_CRT}" | base64 -d - > "${HOME}/k8s_ca.crt"
|
||||
|
||||
# Setup the local kubectl client
|
||||
kubectl config set-context atst-deployer \
|
||||
--cluster=atat-cluster \
|
||||
--user=atat-deployer \
|
||||
--namespace=${NAMESPACE}
|
||||
|
||||
kubectl config set-cluster atat-cluster \
|
||||
--embed-certs=true \
|
||||
--server="${K8S_ENDPOINT}" \
|
||||
--certificate-authority="${HOME}/k8s_ca.crt"
|
||||
|
||||
kubectl config set-credentials atat-deployer --token="$(echo ${K8S_USER_TOKEN} | base64 -d -)"
|
||||
|
||||
kubectl config use-context atst-deployer
|
||||
kubectl config current-context
|
||||
|
||||
# Update the ATST deployment
|
||||
kubectl -n ${NAMESPACE} set image deployment.apps/atst atst="${IMAGE_NAME}"
|
||||
kubectl -n ${NAMESPACE} set image deployment.apps/atst-worker atst-worker="${IMAGE_NAME}"
|
||||
|
||||
# Wait for deployment to finish
|
||||
if ! timeout -t "${MAX_DEPLOY_WAIT}" -s INT kubectl -n ${NAMESPACE} rollout status deployment/atst
|
||||
then
|
||||
# Deploy did not finish before max wait time; abort and rollback the deploy
|
||||
kubectl -n ${NAMESPACE} rollout undo deployment/atst
|
||||
kubectl -n ${NAMESPACE} rollout undo deployment/atst-worker
|
||||
# Exit with a non-zero return code
|
||||
exit 2
|
||||
fi
|
||||
@ -1,240 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: atat
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
labels:
|
||||
app: atst
|
||||
name: atst
|
||||
namespace: atat
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
role: web
|
||||
replicas: 2
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: atst
|
||||
role: web
|
||||
spec:
|
||||
securityContext:
|
||||
fsGroup: 101
|
||||
containers:
|
||||
- name: atst
|
||||
image: registry.atat.code.mil:443/atst-prod:bf4eb235
|
||||
resources:
|
||||
requests:
|
||||
memory: "2500Mi"
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: atst-envvars
|
||||
volumeMounts:
|
||||
- name: atst-config
|
||||
mountPath: "/opt/atat/atst/atst-overrides.ini"
|
||||
subPath: atst-overrides.ini
|
||||
- name: nginx-client-ca-bundle
|
||||
mountPath: "/opt/atat/atst/ssl/server-certs/ca-chain.pem"
|
||||
subPath: client-ca-bundle.pem
|
||||
- name: uwsgi-config
|
||||
mountPath: "/opt/atat/atst/uwsgi-config.ini"
|
||||
subPath: uwsgi-config.ini
|
||||
- name: uwsgi-socket-dir
|
||||
mountPath: "/var/run/uwsgi"
|
||||
- name: atst-nginx
|
||||
image: nginx:alpine
|
||||
ports:
|
||||
- containerPort: 8442
|
||||
name: http
|
||||
- containerPort: 8443
|
||||
name: https
|
||||
volumeMounts:
|
||||
- name: nginx-auth-tls
|
||||
mountPath: "/etc/ssl/private"
|
||||
- name: nginx-client-ca-bundle
|
||||
mountPath: "/etc/ssl/client-ca-bundle.pem"
|
||||
subPath: client-ca-bundle.pem
|
||||
- name: nginx-config
|
||||
mountPath: "/etc/nginx/conf.d/atst.conf"
|
||||
subPath: atst.conf
|
||||
- name: nginx-config
|
||||
mountPath: "/etc/nginx/conf.d/00json_log.conf"
|
||||
subPath: 00json_log.conf
|
||||
- name: nginx-dhparam
|
||||
mountPath: "/etc/ssl/dhparam.pem"
|
||||
subPath: dhparam.pem
|
||||
- name: nginx-htpasswd
|
||||
mountPath: "/etc/nginx/.htpasswd"
|
||||
subPath: .htpasswd
|
||||
- name: uwsgi-socket-dir
|
||||
mountPath: "/var/run/uwsgi"
|
||||
imagePullSecrets:
|
||||
- name: regcred
|
||||
volumes:
|
||||
- name: atst-config
|
||||
secret:
|
||||
secretName: atst-config-ini
|
||||
items:
|
||||
- key: override.ini
|
||||
path: atst-overrides.ini
|
||||
mode: 0644
|
||||
- name: nginx-auth-tls
|
||||
secret:
|
||||
secretName: auth-atst-ingress-tls
|
||||
items:
|
||||
- key: tls.crt
|
||||
path: auth.atat.crt
|
||||
mode: 0644
|
||||
- key: tls.key
|
||||
path: auth.atat.key
|
||||
mode: 0640
|
||||
- name: nginx-client-ca-bundle
|
||||
secret:
|
||||
secretName: nginx-client-ca-bundle
|
||||
items:
|
||||
- key: client-ca-bundle.pem
|
||||
path: client-ca-bundle.pem
|
||||
mode: 0666
|
||||
- name: nginx-config
|
||||
configMap:
|
||||
name: atst-nginx
|
||||
items:
|
||||
- key: nginx-config
|
||||
path: atst.conf
|
||||
- key: nginx-json-log-config
|
||||
path: 00json_log.conf
|
||||
- name: nginx-dhparam
|
||||
secret:
|
||||
secretName: dhparam-4096
|
||||
items:
|
||||
- key: dhparam.pem
|
||||
path: dhparam.pem
|
||||
mode: 0640
|
||||
- name: nginx-htpasswd
|
||||
secret:
|
||||
secretName: atst-nginx-htpasswd
|
||||
items:
|
||||
- key: htpasswd
|
||||
path: .htpasswd
|
||||
mode: 0640
|
||||
- name: uwsgi-config
|
||||
configMap:
|
||||
name: atst-config
|
||||
items:
|
||||
- key: uwsgi-config
|
||||
path: uwsgi-config.ini
|
||||
mode: 0644
|
||||
- name: uwsgi-socket-dir
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
labels:
|
||||
app: atst
|
||||
name: atst-worker
|
||||
namespace: atat
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
role: worker
|
||||
replicas: 1
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: atst
|
||||
role: worker
|
||||
spec:
|
||||
securityContext:
|
||||
fsGroup: 101
|
||||
containers:
|
||||
- name: atst-worker
|
||||
image: registry.atat.code.mil:443/atst-prod:bf4eb235
|
||||
args: ["/bin/bash", "-c", "/opt/atat/atst/script/rq_worker"]
|
||||
resources:
|
||||
requests:
|
||||
memory: "500Mi"
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: atst-envvars
|
||||
- configMapRef:
|
||||
name: atst-worker-envvars
|
||||
volumeMounts:
|
||||
- name: atst-config
|
||||
mountPath: "/opt/atat/atst/atst-overrides.ini"
|
||||
subPath: atst-overrides.ini
|
||||
imagePullSecrets:
|
||||
- name: regcred
|
||||
volumes:
|
||||
- name: atst-config
|
||||
secret:
|
||||
secretName: atst-config-ini
|
||||
items:
|
||||
- key: override.ini
|
||||
path: atst-overrides.ini
|
||||
mode: 0644
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels:
|
||||
app: atst
|
||||
name: atst
|
||||
namespace: atat
|
||||
spec:
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: 8442
|
||||
selector:
|
||||
role: web
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels:
|
||||
app: atst
|
||||
name: atst-auth
|
||||
namespace: atat
|
||||
spec:
|
||||
type: NodePort
|
||||
ports:
|
||||
- name: https
|
||||
protocol: TCP
|
||||
nodePort: 32751
|
||||
port: 8443
|
||||
selector:
|
||||
role: web
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: atst
|
||||
namespace: atat
|
||||
annotations:
|
||||
kubernetes.io/tls-acme: "true"
|
||||
kubernetes.io/ingress.class: "nginx"
|
||||
nginx.ingress.kubernetes.io/proxy-body-size: 10m
|
||||
spec:
|
||||
tls:
|
||||
- secretName: atst-ingress-tls
|
||||
hosts:
|
||||
- atat.code.mil
|
||||
- www.atat.code.mil
|
||||
rules:
|
||||
- host: www.atat.code.mil
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
backend:
|
||||
serviceName: atst
|
||||
servicePort: 80
|
||||
@ -1,4 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
kubectl -n atat delete secret atst-config-ini
|
||||
kubectl -n atat create secret generic atst-config-ini --from-file="${1}"
|
||||
@ -1,4 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
kubectl -n atat delete secret atst-config-ini
|
||||
kubectl -n atat create secret generic nginx-client-ca-bundle --from-file="${1}"
|
||||
@ -1,4 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
kubectl -n atat delete secret dhparam-4096
|
||||
kubectl -n atat create secret generic dhparam-4096 --from-file="${1}"
|
||||
@ -1,4 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
kubectl -n atat delete secret atst-nginx-htpasswd
|
||||
kubectl -n atat create secret generic atst-nginx-htpasswd --from-file="${1}"
|
||||
@ -1,35 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: atst-config
|
||||
namespace: atat
|
||||
data:
|
||||
uwsgi-config: |-
|
||||
[uwsgi]
|
||||
callable = app
|
||||
module = app
|
||||
socket = /var/run/uwsgi/uwsgi.socket
|
||||
plugin = python3
|
||||
plugin = logfile
|
||||
virtualenv = /opt/atat/atst/.venv
|
||||
chmod-socket = 666
|
||||
|
||||
; logger config
|
||||
|
||||
; application logs: log without modifying
|
||||
logger = secondlogger stdio
|
||||
log-route = secondlogger atst
|
||||
log-encoder = format:secondlogger ${msg}
|
||||
|
||||
; default uWSGI messages (start, stop, etc.)
|
||||
logger = default stdio
|
||||
log-route = default ^((?!atst).)*$
|
||||
log-encoder = json:default {"timestamp":"${strftime:%%FT%%T}","source":"uwsgi","severity":"DEBUG","message":"${msg}"}
|
||||
log-encoder = nl
|
||||
|
||||
; uWSGI request logs
|
||||
logger-req = stdio
|
||||
log-format = request_id=%(var.HTTP_X_REQUEST_ID), pid=%(pid), remote_add=%(addr), request=%(method) %(uri), status=%(status), body_bytes_sent=%(rsize), referer=%(referer), user_agent=%(uagent), http_x_forwarded_for=%(var.HTTP_X_FORWARDED_FOR)
|
||||
log-req-encoder = json {"timestamp":"${strftime:%%FT%%T}","source":"req","severity":"INFO","message":"${msg}"}
|
||||
log-req-encoder = nl
|
||||
@ -1,10 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: atst-worker-envvars
|
||||
namespace: atat
|
||||
data:
|
||||
TZ: UTC
|
||||
DISABLE_CRL_CHECK: "True"
|
||||
CRL_STORAGE_PROVIDER: CLOUDFILES
|
||||
Loading…
x
Reference in New Issue
Block a user