Jump to table of contents

Debug Test Failures

When a test fails it means that a recent change has broken the application. The log files for the tests will will indicate which tests failed and will list all of the commands that were run before the failure occurred.

In most cases, the server’s error log will contain important clues about what went wrong.

Debug locally #

Cypress outputs the log directly to the terminal. When the tests are run from the Cypress UI additional information on the browser’s state will be available.

When the tests are run from the terminal you will find a screenshot of the browser’s state when the test failed in cypress/screenshots.

Debug CI #

The CI tests can be difficult to debug. Travis provides access to its build logs and screenshots of test failures to help.

Short video showing how to access log of build results.

This provides a log of the entire build process including the logs from the Cypress and PHP Unit tests. Parts of the log, such as the server’s error log, are truncated and can only be reached by accessing the raw log.

Short video showing how to access the raw log of build results.

When viewing the raw log, run a search in your browser for “fatal”. This is a quick way to find the source of many – but not all – test failures.

Travis debug mode #

Travis provides a debug mode. When enabled it is possible to log into the Travis environment via SSH and explore the tests manually.

Screenshots #

Screenshots can be accessed by using the uuencode tool on the Travis VM. To extract the screenshots related to a test failure, follow these steps.

  • When viewing the test output on Travis, click the “Raw Log” button.
  • Save the log to your computer, for example /tmp/log.txt.
  • Run cat /tmp/log.txt | uudecode | tar xvz to extract the screenshots locally.

If you are struggling to understand a test failure in Travis, it may be best to run the tests locally where you have more insight into the state of the application.

Learn how to write tests for plugins and themes.