Add script for running Ghost Inspector tests locally.
Eventually, this should replace the CircleCI config for running the integration tests to avoid duplication. In the interest of time so that I don't have to debug broken builds, I'm only adding it as a utility script.
This commit is contained in:
dandds
parent
95697db8f5
commit
0851e42572
94
script/integration_tests
Executable file
94
script/integration_tests
Executable file
@ -0,0 +1,94 @@
|
||||
#!/bin/bash
|
||||
|
||||
# script/integration_tests: Run the integration tests via docker.
|
||||
set -e
|
||||
|
||||
if [ -z "${CONTAINER_TIMEOUT+is_set}" ]; then
|
||||
CONTAINER_TIMEOUT=200
|
||||
fi
|
||||
|
||||
# Expected settings. Script will error if these are not provided.
|
||||
SETTINGS=(
|
||||
CONTAINER_IMAGE
|
||||
NGROK_TOKEN
|
||||
GI_API_KEY
|
||||
GI_SUITE
|
||||
)
|
||||
|
||||
# Loop all expected settings. Track ones that are missing. If any
|
||||
# are missing, exit.
|
||||
MISSING_SETTINGS=()
|
||||
for envvar in "${SETTINGS[@]}"; do
|
||||
if [ -z "${!envvar}" ]; then
|
||||
MISSING_SETTINGS+=(${envvar})
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ ${#MISSING_SETTINGS[@]} > 0 ]]; then
|
||||
>&2 echo "The following variables need to be set:"
|
||||
for missing in "${MISSING_SETTINGS[@]}"; do
|
||||
>&2 echo $missing
|
||||
done
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Remove any existing container and network instances
|
||||
docker container stop redis postgres test-atat || true && docker container rm redis postgres test-atat || true
|
||||
docker network rm atat || true
|
||||
|
||||
# Create network
|
||||
docker network create atat
|
||||
|
||||
# Start Redis and Postgres
|
||||
docker run -d --network atat --link redis:redis -p 6379:6379 --name redis circleci/redis:4-alpine3.8
|
||||
docker run -d --network atat --link postgres:postgres -p 5432:5432 --name postgres circleci/postgres:10-alpine-ram
|
||||
# Wait for datastores to be available
|
||||
sleep 3
|
||||
|
||||
# Create database and run migrations
|
||||
docker exec postgres createdb -U postgres atat
|
||||
docker run --network atat -e PGDATABASE=atat -e PGHOST=postgres -e REDIS_HOST=redis:6379 $CONTAINER_IMAGE .venv/bin/python .venv/bin/alembic upgrade head
|
||||
docker run --network atat -e PGDATABASE=atat -e PGHOST=postgres -e REDIS_HOST=redis:6379 $CONTAINER_IMAGE .venv/bin/python script/seed_roles.py
|
||||
|
||||
# Start application container
|
||||
docker run -d \
|
||||
-e DISABLE_CRL_CHECK=true \
|
||||
-e PGHOST=postgres \
|
||||
-e REDIS_HOST=redis:6379 \
|
||||
-p 8000:8000 \
|
||||
--network atat \
|
||||
--name test-atat \
|
||||
$CONTAINER_IMAGE \
|
||||
/bin/sh -c "
|
||||
echo CLOUD_PROVIDER=mock > .env &&\
|
||||
yarn build &&\
|
||||
uwsgi \
|
||||
--callable app \
|
||||
--module app \
|
||||
--plugin python3 \
|
||||
--virtualenv /install/.venv \
|
||||
--http-socket :8000
|
||||
"
|
||||
|
||||
# Use curl to wait for application container to become available
|
||||
docker pull curlimages/curl:latest
|
||||
docker run --network atat \
|
||||
curlimages/curl:latest \
|
||||
curl --connect-timeout 3 \
|
||||
--max-time 5 \
|
||||
--retry $CONTAINER_TIMEOUT \
|
||||
--retry-connrefused \
|
||||
--retry-delay 1 \
|
||||
--retry-max-time $CONTAINER_TIMEOUT \
|
||||
test-atat:8000
|
||||
|
||||
# Run Ghost Inspector tests
|
||||
docker pull ghostinspector/test-runner-standalone:latest
|
||||
docker run \
|
||||
-e NGROK_TOKEN=$NGROK_TOKEN \
|
||||
-e GI_API_KEY=$GI_API_KEY \
|
||||
-e GI_SUITE=$GI_SUITE \
|
||||
-e GI_PARAMS_JSON='{}' \
|
||||
-e APP_PORT="test-atat:8000" \
|
||||
--network atat \
|
||||
ghostinspector/test-runner-standalone:latest
|
||||
@ -30,7 +30,7 @@ QA lead monitors all failures on the CI suite and determines that such a change
|
||||
the Ghost Inspector UI.
|
||||
|
||||
3. If the (potentially) failing test will need to be reworked to account for functional changes, the test is moved from the CI
|
||||
suite to the "Holding" suite until the PR is merged. Then the test can be edited and returned to the CI suite.
|
||||
suite to the "Holding" suite until the PR is merged. Then the test can be edited and returned to the CI suite.
|
||||
|
||||
## Running Ghost Inspector tests locally
|
||||
|
||||
@ -38,26 +38,26 @@ To run the Ghost Inspector tests against a local instance of AT-AT,
|
||||
you will need the following:
|
||||
|
||||
- [docker](https://docs.docker.com/v17.12/install/)
|
||||
- [circleci CLI tool](https://circleci.com/docs/2.0/local-cli/#installation)
|
||||
- the prerequisite variable information listed [here](https://ghostinspector.com/docs/integration/circle-ci/)
|
||||
- the prerequisite variable information listed [here](https://ghostinspector.com/docs/integration/circle-ci/): NGROK_TOKEN, GI_API_KEY, GI_SUITE
|
||||
|
||||
The version of our CircleCI config (2.1) is incompatible with the
|
||||
`circleci` tool. First run:
|
||||
First you will need to build a copy of the container:
|
||||
|
||||
```
|
||||
circleci config process .circleci/config.yml > local-ci.yml
|
||||
docker build . --build-arg CSP=azure -f ./Dockerfile -t atat:builder --target builder
|
||||
```
|
||||
|
||||
Then run the job:
|
||||
This builds the first stage of the docker container, which is the one we need to run integration tests. You can tag the container whatever you want; in the example we've tagged it "atat:builder".
|
||||
|
||||
Then you can run the integration tests script. You will need four environment variables set: the three mentioned previously and CONTAINER_IMAGE. You can either export them or set them inline in the command you use to run the script. In the example we'll set them inline:
|
||||
|
||||
```
|
||||
circleci local execute -e GI_SUITE=<SUITE_ID> -e GI_API_KEY=<API KEY> -e NGROK_TOKEN=<NGROK TOKEN> --job integration-tests -c local-ci.yml
|
||||
NGROK_TOKEN=<token> GI_API_KEY=<api key> GI_SUITE=<suite> CONTAINER_IMAGE=atat:builder ./script/integration_tests
|
||||
```
|
||||
|
||||
If the job fails and you want to re-run it, you may receive errors
|
||||
about running docker containers or the network already existing.
|
||||
Some version of the following should reset your local docker state:
|
||||
### Troubleshooting
|
||||
|
||||
```
|
||||
docker container stop redis postgres test-atat; docker container rm redis postgres test-atat ; docker network rm atat
|
||||
```
|
||||
- If you get errors regarding ports being in use, make sure you don't have instances of the Flask app, Postgres, or Redis running locally using those ports.
|
||||
- If the curl command used to wait for the application container times out and fails, you can increase the timeout by setting a CONTAINER_TIMEOUT environment variable. It defaults to 200 in the script.
|
||||
- The curl command will print errors until it successfully connects to the application container. These are normal and expected. When it finally connects, it will print the ATAT home page HTML to STDOUT.
|
||||
- You may see errors like "No such container". The script attempts to clean up any previous incarnations of the containers before it starts, and it may print errors when it doesn't find them. This is fine.
|
||||
- The script is, for the most part, a series of docker commands, so try running the commands individually and debugging that way.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user