From e7cdcd349712a7ca757616a153a56edcae4b2b30 Mon Sep 17 00:00:00 2001 From: "Jay R. Newlin (PromptWorks)" Date: Wed, 27 Nov 2019 16:58:21 -0500 Subject: [PATCH] Add section on dealing with changes to the UI and failing tests --- uitests/README.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/uitests/README.md b/uitests/README.md index 1ab5777a..5a119302 100644 --- a/uitests/README.md +++ b/uitests/README.md @@ -1,6 +1,6 @@ # Ghost Inspector Readme -The suite of Ghost Inspector tests that runs as part of the CI/CD workflow +The suite of Ghost Inspector tests that runs as part of the CI workflow can be found [here](https://app.ghostinspector.com/suites/5d9e3d303ab5d56633c11598). Its current status is https://api.ghostinspector.com/v1/suites/5d9e3d303ab5d56633c11598/status-badge [Another suite](https://app.ghostinspector.com/suites/5d9603f1af31210914da04ca) of Ghost Inspector tests runs once daily against the Staging site. Status: https://api.ghostinspector.com/v1/suites/5d9603f1af31210914da04ca/status-badge @@ -17,6 +17,21 @@ for adding a second CLIN to a TO. These tests rely on Ghost Inspector's "Import necessary setup for the current test. This also ensures that tests can be run in any sequence because Ghost Inspector launches up to four tests simultaneously. +## Handling UI changes + +As with any UI-testing system, Ghost Inspector tests will fail because of changes to the UI. This can be problematic since failures +cause errors in the CI workflow, which could cause the Pull Request not to be deployed. To mitigate this issue, we have utilized +the following strategies: + +1. Front-end developers notify the project QA lead of changes that will potentially cause Ghost Inspector failures --OR-- the +QA lead monitors all failures on the CI suite and determines that such a change has occurred. + +2. If only one step in a test is affected (e.g. a visual but not functional change), that step can be marked as "optional" in +the Ghost Inspector UI. + +3. If the (potentially) failing test will need to be reworked to account for functional changes, the test is moved from the CI +suite to the "Holding" suite until the PR is merged. Then the test can be edited and returned to the CI suite. + ## Running Ghost Inspector tests locally To run the Ghost Inspector tests against a local instance of AT-AT,