Testing

Owned by Paul McCullough (Unlicensed)
Last updated: Feb 14, 2019 by Samuel Valdez (Unlicensed)
2 min read

1. User Interface Testing

This check list seems to catch most bugs. We are not going to automate UI testing because it's not appropriate. The application is not massive and is scheduled to be mature in another 12 months or so. Just be sure to execute all the tests therein which takes about an hour if you are adequately caffeinated and focused.

2. Unit Testing

Overview

There are a modest number of Python/Django unit tests. However, there are currently no unit tests for the JavaScript code. Tests are easy to write and help to prevent regression errors. 

When to write a test

When a bug is found, try to write a test that finds it. If you start the debugger, consider writing a unit test test instead. This can be expensive and so it is reasonable to debate this point.

Running the unit tests
  1. Connect to the web server of a given environment. (SF_DEV, SF_QA, SF_PROD or SD_PROD)
  2. Navigate to the folder containing the script that launches the unit tests.
    1. cd /var/www/html/eas/
  3. Launch the unit tests as sudo.
    1. sudo ./run_tests.sh
    2. This goes through settings.py and calls the tests in https://bitbucket.org/sfgovdt/eas/src/default/web/test_suites.cmd
Reviewing and saving the test results

The units tests take under one minute to execute. Results are output to the console with 'OK' after each test that passes and 'FAILED' for each test that fails. Manually review the results and fix any failed test as needed.

To archive the results copy the console output to the clipboard and paste into a new text file. Upload the file to a folder containing testing-related artifacts for the given release. For example, R:\Tec\...\Eas\_Task\2018_2019\20181220_255_1.3.7\TestResults\EAS Unit Test Results - All Tests Passed - console output - 20190117.txt

3. Testing the Change Notification Emails

The EAS has automated routines that send email about new addresses. Run theses routines as part of the testing process.
  1. Begin by confirming that you are a recipient of the Change Notification emails.
    1. See config_notifications.py in /automation/src/config/live/ and check/modify variables devAddressChangeNotificationEmails and/or prodAddressChangeNotificationEmails so that they include your email address.
    2. For environments SF_DEV and SF_QA see the devAddressChangeNotificationEmails and prodSupportEmails variables. For environments SF_PROD and SD_PROD see the prodAddressChangeNotificationEmails and prodAnnounceNewReleaseEmails variables.
  2. Connect to a Windows automation server (SF_DEV, SF_QA, SF_PROD or SD_PROD) using Microsoft Remote Desktop.
  3. Open 'Command Prompt' and change to the folder containing the change notification script https://bitbucket.org/sfgovdt/eas/src/default/automation/src/job.py
    • C:\apps\eas_automation\automation\src
  4. Run the address_change_report job: 
    1. python job.py --env <environment> --job address_change_report --action EXECUTE --v
    2. Substitute <environment> with one of the relevant environments: SF_DEV, SF_QA, SF_PROD, SD_PROD
  5. Run the address_change_notification job:
    1. python job.py --env <environment> --job address_change_notification --action EXECUTE --v
    2.  Substitute <environment> with one of the relevant environments: SF_DEV, SF_QA, SF_PROD, SD_PROD
  6. Run the announce_new_release job:
    1. python job.py --env <environment> --job announce_new_release --action EXECUTE --v
    2. Substitute <environment> with one of the relevant environments: SF_DEV, SF_QA, SF_PROD, SD_PROD
  7. Check your email inbox to confirm that the change notifications emails were received.
  8. If applicable, revert any temporary changes to the file config_notifications.py that were made in Step 1.