-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
move pytest example files from pytest-test branch
- Loading branch information
Showing
3 changed files
with
182 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,121 @@ | ||
# Pytest Documentation | ||
|
||
### Installing Pytest | ||
|
||
**Note:** Before continuing, ensure that a python enviroment has been set up and his currently running. | ||
View [this](https://github.itap.purdue.edu/ECN/webqueue2/blob/master/Dev%20Environment%20Setup%20Guide.md#step-4-setup-python-for-api) page for more information on setting up the python environment | ||
|
||
|
||
Run the following command to install pytest: `pip install pytest`. | ||
|
||
### Using Pytest | ||
|
||
Now that pytest is installed, you will want to create a test script. The test script can be used for testing any python script, however any test script that is created should, for simplicity, be created in the same directory as the target test script, and it should import the desired python script to test. For example, a pytest script might look something like this: | ||
|
||
```python | ||
import script.py | ||
|
||
def test_a_function(): | ||
assert script.subtract(3, 2) == 1 | ||
assert script.subtract(4, 6) == -2 | ||
``` | ||
|
||
Where `script.py` is the script that needs to be tested and `subtract` is a defined function in `script.py`. | ||
|
||
It is recomended to name the test script and append `test_` to the begining of the file name so it looks like this: `test_pythonTestingScript.py`. When running the `pytest` command, pytest will automatcialy check the current directory and sub directories for `.py` files begining with filenames begining with `test_`. If multiple files have the `test_` prefix, pytest will only test the first script encountered aphabetically. Since multiple testing scripts maybe present in the `pytest` folder, you can also pass the file name of the testing script as an argument when running `pytest` from the terminal. | ||
|
||
To make effective use of pytest, your testing script should test individual functions. To do this, create a new function and name it so that it begins with `test_` like this: | ||
|
||
```python | ||
def test_function_to_test(): | ||
``` | ||
|
||
Pytest will individually test each function that follows this syntax. | ||
|
||
#### Assertions | ||
|
||
Pytest relies on python assertions to function correctly. Assertions in python are used to test a condition, and if that condition fails, then the script exits with an error. Pytest is very verbose when encountering an error so when the assertion error is encountered, it is very easy to identify the reason for the error as well as the values associated with throwing the error. To use assertions, simply place the word `assert` in front of any condition. Some example assertions might include: | ||
|
||
```python | ||
assert 1==1 | ||
assert "hello" in "hello there!" | ||
assert varOne >= varTwo | ||
assert arrayOne[1] == arrayTwo [4] | ||
``` | ||
|
||
The conditions used for assertions should be conditions that would function in any python `if` statement. | ||
|
||
### Running Pytest | ||
|
||
After developing a testing script, you can run the test script using `pytest` through the terminal by using this command: `pytest [path_to_test_script]`. Pytest isn't a module that has to be imported directly into a python script but ratehr a command that can be executied in the terminal, importing pytest into a script however, does add some functionality. Pytest will test each function denoted with the `test_` syntax. If an error is encountered, pytest will report the line that failed along with the values that caused the line to fail, however pytest will not test anymore lines within that function until the error is resolved. If there is more than one function with the `test_` syntax, pytest will test those functions as well, even if any of the previous functions encountered an error. Here is some example output: | ||
|
||
```bash | ||
======================================================================= FAILURES ======================================================================== | ||
_______________________________________________________________________ test_num ________________________________________________________________________ | ||
|
||
def test_num(): | ||
assert int(5) == 5 | ||
assert 5 + 5 == 10 | ||
assert 1 == int('1') | ||
> assert int("five") == 5 | ||
E ValueError: invalid literal for int() with base 10: 'five' | ||
|
||
pytester.py:6: ValueError | ||
================================================================ short test summary info ================================================================ | ||
FAILED pytester.py::test_num - ValueError: invalid literal for int() with base 10: 'five' | ||
=================================================================== 1 failed in 0.04s =================================================================== | ||
``` | ||
|
||
### Additional Features | ||
|
||
#### Markers | ||
|
||
Markers are in-line code that can denote different functions. To make use of these markers, the script must `import pytest` and follow the syntax `@pytest.mark.marker_name`. Executing pytest with the `-m` flag followed by the `marker_name` will make pytest test only the function that the marker is directly above (See `test_example_pytest.py` for marker examples). | ||
It is acceptable to have two markers with the same name, if multiple markers with the same name are present, pytest will run all functions with the coresponding marker name passed as an arguement in the terminal. | ||
**Note:** Pytest will report a warning with each marker that is used in a script. Any warnings regarding markers can be safley ignored and/or disabled. | ||
|
||
#### Verbose | ||
|
||
The `-v` flag (or alternatively `--verbose`) will cause pytest to print all functions that are tested, including functions that did not throw errors, which is useful to explicitly view what functions "failed" the pytest as well as the functions that "passed". | ||
Example additional output from the verbose flag: | ||
|
||
```bash | ||
pytester.py::test_num FAILED [ 25%] | ||
pytester.py::test_string FAILED [ 50%] | ||
pytester.py::test_array PASSED [ 75%] | ||
pytester.py::test_dictionary FAILED [100%] | ||
``` | ||
|
||
#### Test Specific Functions | ||
|
||
In addition to markers, pytest can specific function in a script if it is passed with the file name like this: | ||
|
||
```bash | ||
pytest test_script.py::test_function | ||
``` | ||
|
||
#### Skip testing on functions | ||
|
||
Sometimes it might be useful, or even necessary, to skip over the testing of a particular function. To do this, use `@pytest.mark.skip(reason="reason_for_skipping")` like you would use a normal marker. Pytest will skip over the function that this marker is above. If the `-v` flag is passed in the terminal, the function will be labled as skipped. Additionally, an `if` clause can be used to determine if the function should be skipped with this syntax: `@pytest.mark.skipif([logical_condition], reason="reason_for_skipping")`, where pytest will not test the function if the logical condition is true. | ||
|
||
#### Print statements | ||
|
||
By default, pytest will not print output from print statements to the terminal unless a function fails. To enable print statements to output to the terminal, pass the `-s` flag with the pytest command. | ||
|
||
#### Custom Pytest Configurations | ||
|
||
Pytest can be configured py use of `conftest.py` files, which add additional functionality to pytest. (See the `conftest.py` file in the pytest directory). Pytest will automatically look for any file named `conftest.py` before executing the testing script and apply any of the configurations within the file before, during, or after pytest execution. In the current use of the `conftest.py` file, pytest will output the name of any function that fails testing into a file named `pytest_failures`. The example contents of `pytest_failures` after running pytest might look something like this: | ||
``` | ||
test_example_pytester.py::test_numextra | ||
test_example_pytester.py::test_stringextra | ||
test_example_pytester.py::test_arrayextra | ||
test_example_pytester.py::test_dictionaryextra | ||
``` | ||
More information on `conftest.py` and configurations for pytest can be found in the webpages listed below: | ||
https://docs.pytest.org/en/latest/example/simple.html#post-process-test-reports-failures | ||
https://docs.pytest.org/en/2.7.3/plugins.html?highlight=re | ||
|
||
### Pytest Tutoiral | ||
|
||
Much of the information in this README comes directly from this tutorial, it is very indepth, but it is also very easy to understand. | ||
https://www.youtube.com/watch?v=bbp_849-RZ4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# https://docs.pytest.org/en/latest/example/simple.html | ||
|
||
import pytest | ||
import os.path | ||
|
||
failuresFilePath = "pytest_failures" | ||
|
||
if os.path.exists(failuresFilePath): | ||
os.remove(failuresFilePath) | ||
|
||
@pytest.hookimpl(tryfirst=True, hookwrapper=True) | ||
def pytest_runtest_makereport(item, call): | ||
|
||
# execute all other hooks to obtain the report object | ||
outcome = yield | ||
rep = outcome.get_result() | ||
|
||
# we only look at actual failing test calls, not setup/teardown | ||
if rep.when == "call" and rep.failed: | ||
mode = "a" | ||
with open(failuresFilePath, mode) as f: | ||
f.write(rep.nodeid + "extra" + "\n") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
#Importing pytest is not required unless using markers, as demonstrated below | ||
import pytest | ||
|
||
@pytest.mark.num | ||
def test_num(): | ||
assert int(5) == 5 | ||
assert 5 + 5 == 10 | ||
assert 1 == int('1') | ||
assert int("five") == 5 | ||
|
||
@pytest.mark.string | ||
def test_string(): | ||
assert "Hello World!" == "Hello World!" | ||
assert "Hello" in "Hello World" | ||
assert "Hello" == "He" + "llo" | ||
assert "Hello" == "World" | ||
|
||
@pytest.mark.array | ||
def test_array(): | ||
testArrayOne = [1, 2, 3, 4, 5] | ||
testArrayTwo = [5, 4, 3, 2, 1] | ||
testArrayThree = ["1", "2", "3", "4", "5"] | ||
assert testArrayOne[2] == testArrayTwo[2] | ||
assert int(testArrayThree[0]) == testArrayOne[0] | ||
assert testArrayTwo[1] - testArrayOne[4] == -1 | ||
assert testArrayOne[1] == testArrayThree[1] | ||
|
||
@pytest.mark.dict | ||
def test_dictionary(): | ||
testDict = { | ||
"number": 1, | ||
"color": "red", | ||
"shape": "square", | ||
"letter": "A" | ||
} | ||
assert testDict['number'] == 1 | ||
assert 'color' in testDict | ||
assert testDict.get('letter') == "A" | ||
assert testDict['number'] + testDict['number'] == 1 |