pk/atst
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #1222 from dod-ccpo/ghost-inspector-readme

Browse Source 
Create Ghost Inspector README
This commit is contained in:
Jay R. Newlin (PromptWorks) 2019-12-02 13:08:14 -05:00 committed by GitHub
commit 31b7629415
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 65 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
README.md
View File

@ -270,33 +270,8 @@ execute UI tests than vanilla Selenium. Ghost Inspector tests and steps can
be exported to files that the Selenium IDE can import. We export these tests/steps
regularly and archive them with the AT-AT codebase in the `uitests` directory.
To run the Ghost Inspector tests against a local instance of AT-AT,
you will need the following:
- [docker](https://docs.docker.com/v17.12/install/)
- [circleci CLI tool](https://circleci.com/docs/2.0/local-cli/#installation)
- the prerequisite variable information listed [here](https://ghostinspector.com/docs/integration/circle-ci/)
The version of our CircleCI config (2.1) is incompatible with the
`circleci` tool. First run:
```
circleci config process .circleci/config.yml > local-ci.yml
```
Then run the job:
```
circleci local execute -e GI_SUITE=<SUITE_ID> -e GI_API_KEY=<API KEY> -e NGROK_TOKEN=<NGROK TOKEN> --job integration-tests -c local-ci.yml
```
If the job fails and you want to re-run it, you may receive errors
about running docker containers or the network already existing.
Some version of the following should reset your local docker state:
```
docker container stop redis postgres test-atat; docker container rm redis postgres test-atat ; docker network rm atat
```
For further information about Ghost Inspector and its use in AT-AT, check out [its README](./uitests/README.md)
in the `uitests` directory.
## Notes

63
uitests/README.md Normal file
View File

@ -0,0 +1,63 @@
# Ghost Inspector Readme
The suite of Ghost Inspector tests that runs as part of the CI workflow
can be found [here](https://app.ghostinspector.com/suites/5d9e3d303ab5d56633c11598). Its current status is https://api.ghostinspector.com/v1/suites/5d9e3d303ab5d56633c11598/status-badge
[Another suite](https://app.ghostinspector.com/suites/5d9603f1af31210914da04ca) of Ghost Inspector tests runs once daily against the Staging site. Status: https://api.ghostinspector.com/v1/suites/5d9603f1af31210914da04ca/status-badge
## Testing philosophy
The tests have been created to traverse the most common user flows in AT-AT. There are a few tests (e.g. "New Portfolio - no optional fields")
that check for regressions. Others (e.g. "Remove Portfolio Member") check less-common, "negative path" flows. Tests are added as necessary
to ensure fairly thorough checking of AT-AT.
The tests are constructed in a "stepwise" fashion; that is, no individual test depends upon another, and each test checks one step in complex
user flows. As an example, there are six tests for creating a new Task Order, one corresponding to each screen in the process plus one
for adding a second CLIN to a TO. Each test relies on Ghost Inspector's "Import steps from Test X" functionality to perform all the
necessary setup before running its own check. This also ensures that tests can be run in any sequence since Ghost Inspector launches up to
four tests simultaneously.
## Handling UI changes
As with any UI-testing system, Ghost Inspector tests will fail because of changes to the UI. This can be problematic since failures
cause errors in the CI workflow, which could cause the Pull Request not to be deployed. To mitigate this issue, we have utilized
the following strategies:
1. Front-end developers notify the project QA lead of changes that will potentially cause Ghost Inspector failures --OR-- the
QA lead monitors all failures on the CI suite and determines that such a change has occurred.
2. If only one step in a test is affected (e.g. a visual but not functional change), that step can be marked as "optional" in
the Ghost Inspector UI.
3. If the (potentially) failing test will need to be reworked to account for functional changes, the test is moved from the CI
suite to the "Holding" suite until the PR is merged. Then the test can be edited and returned to the CI suite.
## Running Ghost Inspector tests locally
To run the Ghost Inspector tests against a local instance of AT-AT,
you will need the following:
- [docker](https://docs.docker.com/v17.12/install/)
- [circleci CLI tool](https://circleci.com/docs/2.0/local-cli/#installation)
- the prerequisite variable information listed [here](https://ghostinspector.com/docs/integration/circle-ci/)
The version of our CircleCI config (2.1) is incompatible with the
`circleci` tool. First run:
```
circleci config process .circleci/config.yml > local-ci.yml
```
Then run the job:
```
circleci local execute -e GI_SUITE=<SUITE_ID> -e GI_API_KEY=<API KEY> -e NGROK_TOKEN=<NGROK TOKEN> --job integration-tests -c local-ci.yml
```
If the job fails and you want to re-run it, you may receive errors
about running docker containers or the network already existing.
Some version of the following should reset your local docker state:
```
docker container stop redis postgres test-atat; docker container rm redis postgres test-atat ; docker network rm atat
```