pk/atst
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add section on dealing with changes to the UI and failing tests

Browse Source
This commit is contained in:
Jay R. Newlin (PromptWorks) 2019-11-27 16:58:21 -05:00
parent 83d9000760
commit e7cdcd3497
1 changed files with 16 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
uitests/README.md
View File

@ -1,6 +1,6 @@
# Ghost Inspector Readme # Ghost Inspector Readme
The suite of Ghost Inspector tests that runs as part of the CI/CD workflow The suite of Ghost Inspector tests that runs as part of the CI workflow
can be found [here](https://app.ghostinspector.com/suites/5d9e3d303ab5d56633c11598). Its current status is https://api.ghostinspector.com/v1/suites/5d9e3d303ab5d56633c11598/status-badge can be found [here](https://app.ghostinspector.com/suites/5d9e3d303ab5d56633c11598). Its current status is https://api.ghostinspector.com/v1/suites/5d9e3d303ab5d56633c11598/status-badge
[Another suite](https://app.ghostinspector.com/suites/5d9603f1af31210914da04ca) of Ghost Inspector tests runs once daily against the Staging site. Status: https://api.ghostinspector.com/v1/suites/5d9603f1af31210914da04ca/status-badge [Another suite](https://app.ghostinspector.com/suites/5d9603f1af31210914da04ca) of Ghost Inspector tests runs once daily against the Staging site. Status: https://api.ghostinspector.com/v1/suites/5d9603f1af31210914da04ca/status-badge
@ -17,6 +17,21 @@ for adding a second CLIN to a TO. These tests rely on Ghost Inspector's "Import
necessary setup for the current test. This also ensures that tests can be run in any sequence because Ghost Inspector launches up to four necessary setup for the current test. This also ensures that tests can be run in any sequence because Ghost Inspector launches up to four
tests simultaneously. tests simultaneously.
## Handling UI changes
As with any UI-testing system, Ghost Inspector tests will fail because of changes to the UI. This can be problematic since failures
cause errors in the CI workflow, which could cause the Pull Request not to be deployed. To mitigate this issue, we have utilized
the following strategies:
1. Front-end developers notify the project QA lead of changes that will potentially cause Ghost Inspector failures --OR-- the
QA lead monitors all failures on the CI suite and determines that such a change has occurred.
2. If only one step in a test is affected (e.g. a visual but not functional change), that step can be marked as "optional" in
the Ghost Inspector UI.
3. If the (potentially) failing test will need to be reworked to account for functional changes, the test is moved from the CI
suite to the "Holding" suite until the PR is merged. Then the test can be edited and returned to the CI suite.
## Running Ghost Inspector tests locally ## Running Ghost Inspector tests locally
To run the Ghost Inspector tests against a local instance of AT-AT, To run the Ghost Inspector tests against a local instance of AT-AT,