-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add notes for testing best practices
- Loading branch information
Justin Campbell
committed
Oct 15, 2020
1 parent
3d42ebe
commit ff67aca
Showing
1 changed file
with
86 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
# ["Testing Best Practices (Tutorial)" by: Randy Syring](https://www.youtube.com/watch?v=HH20YPp_5WI&feature=youtu.be) | ||
Video resources are available [on GitHub](https://github.com/rsyring/pyweb-flog). | ||
|
||
# Table of Contents | ||
<!-- TOC --> | ||
|
||
- [Introduction](#introduction) | ||
- [Steps to Make Testing an Asset](#steps-to-make-testing-an-asset) | ||
- [Just Get Started](#just-get-started) | ||
- [Explicit Is Better Than Implicit](#explicit-is-better-than-implicit) | ||
- [Make Tests As Close To the "real world" As Possible](#make-tests-as-close-to-the-real-world-as-possible) | ||
- [Refecator Code for Easier Testing](#refecator-code-for-easier-testing) | ||
- [CLI's Can Test Library Code](#clis-can-test-library-code) | ||
- [Test The Unhappy Paths](#test-the-unhappy-paths) | ||
- [External Dependencies Should Be Mocked](#external-dependencies-should-be-mocked) | ||
- [Tests Are Responsible For Setting State](#tests-are-responsible-for-setting-state) | ||
- [Closing Comments](#closing-comments) | ||
|
||
<!-- /TOC --> | ||
|
||
## Introduction | ||
- Testing can be an asset, liability or expense. | ||
- Tests are assets when they are easy to run, interpret and respond to. This allows for problems to be dealt with in development before code reaches an end user. | ||
- Tests are liabilities when written poorly, producing false negatives and/or test the wrong things. | ||
- Tests are expenses when they take away from the developer working on actual code. | ||
- Time is money so manual, slow, finnicky tests become money sinks whereas automated, quick, robust tests save money. | ||
|
||
## Steps to Make Testing an Asset | ||
Well formed tests should be: | ||
|
||
**Easy to setup:** | ||
- Make dependencies/requirements clear. This can be done with a setup guide, package manager manifest or script. We do this with pip's `requirements.txt` and npm's `package.json`. | ||
- Make environment requirements clear. This can be done with a setup guide, docker file or script. We do this by developing against the known standard of ECN supported Ubuntu hosts. | ||
|
||
**Easy to run:** | ||
- Make tests accessible. We do this by simplifying test running through automated discovery with pytest and jest. | ||
|
||
**Accesible:** | ||
- Where possible, allow unit and integration testing to be run on the developer's machine. We do not current have these types of test but it is what we need to create. The video details using a set of libraries to allow for browser simulation, API mocking and more. | ||
|
||
**Stateless:** | ||
- The success of tests should not depend on the order they're run. | ||
- Each test should clean up after itself so as to not affect the output of the next test. | ||
- The environment (dev, prod; Win, macOS, Ubuntu) should not affect whether a test passes or fails. Logically, this is limited to supported environments. webqueue2 will cannot run on Windows because gunicorn cannot run on Windows, however, webqueue2 will never run on a Windows host so this is ok. | ||
- Use generators instead of static test data to avoid tests failing due to outdated example data. | ||
- Test should be modular, allowing for as broad or specific of testing as needed. (This debugger break points) | ||
|
||
## Just Get Started | ||
The biggest hurdle to testing is starting. Avoid decision fatigue by testing the low haning fruit. | ||
|
||
## Explicit Is Better Than Implicit | ||
When testing, do not hide errors and prevent hiding where it is the default. | ||
|
||
_Ex:_ Handle exceptions in production code, run tests in debug mode. | ||
|
||
## Make Tests As Close To the "real world" As Possible | ||
Automated testing of web applications often runs at the command line, not a browser. Therefore, browser simulators (discussed in the video) become useful as the simulate working with an application as if you were in the browser -- running JavaScript, using User-Agent strings etc. | ||
|
||
## Refecator Code for Easier Testing | ||
Smaller functions are easier to test than large chunks of code so sepearte logical steps into helper functions. | ||
|
||
- In Python this can be dunder functions like `__getMostRecentHeaderByType()`. | ||
- In React this can be adding common functionality to the `/src/utilities.js` file and importing them for use as needed. | ||
|
||
## CLI's Can Test Library Code | ||
Sometimes it is easier to test library code (which should be context agnostic) by running it via the command line instead of in a particular script. It should behave the same given the right data. | ||
|
||
_Ex:_ Bank transaction software cannot have tests run during production because it would actually move money. Instead, the library could be used to mock tests in development without affecting real systems. | ||
|
||
## Test The Unhappy Paths | ||
"Happy paths" are the paths we expect with well formed input. "Unhappy paths" are everything else. For the program to function the happy paths must work but for the program to be robust the unhappy paths needs to be dealt with as well. | ||
|
||
## External Dependencies Should Be Mocked | ||
If your test depends on something you cannot control, the test environment should not rely on that dependency. Instead, it should mock the behavior of the working external dependency. | ||
|
||
_Ex:_ [fake-smtp](http://nilhcem.com/FakeSMTP/) is a program that mocks an SMTP server. Using this in testing allows for a known working SMTP server that will tell us of the problem is in our code or a non-working production SMTP server. | ||
|
||
## Tests Are Responsible For Setting State | ||
The test, not the environment, should check, set and cleanup state to insure the test is testing what it was written to test. Ex: A previous test was supposed to create an object for a subsequent test to use but fails. Now the subsequent test fails despite because its input is bad, not its logic. | ||
|
||
## Closing Comments | ||
- Thinking in tests is a mindset that takes time to learn. | ||
- Test Driven Development is writing tests for the expected behavior _then_ writing the code to conform to that behavior, modelling to code to fit the test. | ||
- Treat tests like code: write it well and keep it updated, improve tests when possible | ||
- Use test coverage tools to see where tests are applied (codecov.io) | ||
- The best time to start is now. |