Unit tests are currently setup to use Pytest.
Setting up an environment for testing is quite simple.
From a clean Linux image you run the following commands, and you'll be all set.
- install docker (perhaps other packages?)
- Download the Sarracenia source
git clone https://github.com/MetPX/sarracenia - Checkout the branch you're working on (optional)
git checkout <BranchName> - Update PIP
pip3 install -U pip - Install base requirements for Sarracenia
pip install -r requirements.txt - Install sarracenia Python modules
pip3 install -e . - Install requirements for PyTest
pip install -r tests/requirements.txt
That's it, you're now ready to run the tests.
From within the repository root directory, simply run pytest tests and you'll see the results. There's a full set of arguments/options that modify the output, outlined here.
As configured, it will output a bit of system information, total tests, followed by a line per file, with dots/letters to indicate the status of each test in that file.
Letter meanings:
(f)ailed, (E)rror, (s)kipped, (x)failed, (X)passed, (p)assed, (P)assed with output, (a)ll except passed (p/P), (w)arnings, (A)ll
Specifying the -v option will make it a bit more verbose, listing each tests, and it's pass/skip/fail status.
Application logs captured during tests can be output with the -o log_cli=true argument.
Basic HTML report of tests can be generated by adding --html=tests/report.html --self-contained-html to the command-line.
Code coverage can be generated by adding --cov-config=tests/.coveragerc --cov=sarracenia --cov-report=xml --cov-report=html to the command-line.
Junit-compatible unit test report can be had with by adding --junitxml=tests/junit/test-results.xml to the command-line.
If you want to run this in VSCode, and have it do all the things nicely, you'll need the following extensions:
- Python
Provides all the smarts for Python development (IntelliSense, linting, debugging, code formatting, refactoring, unit tests, etc..) - Coverage Gutters
Provides a way to track code coverage in VSCode - Python Environment Manager
Not strictly required, but really handy if you're working on a number of Python projects - GitLens — Git supercharged
Not strictly required, but very strongly recommended as it makes VS Code's git features fully functional
Beyond that, changing a few things in your VS Code configs will make it all work.
In settings.json, to get all the reports and coverage when running tests, and allow you to run individual tests even if they have dependencies:
{
"python.testing.pytestArgs": [
"tests", "-v",
"--cov-config=tests/.coveragerc", "--cov=sarracenia", "--cov-report=xml", "--cov-report=html",
"--html=tests/report.html", "--self-contained-html",
"--failed-dependency-action=run", "--missing-dependency-action=run"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true,
"coverage-gutters.coverageBaseDir": "tests/coverage",
}In launch.json (per documentation), to enable full debugging support in your tests:
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Debug Tests",
"type": "python",
"request": "launch",
"program": "${file}",
"purpose": ["debug-test"],
"console": "integratedTerminal",
"justMyCode": false,
"env": {"PYTEST_ADDOPTS": "--no-cov"}
}
]
}NOTE: Don't just squash whatever you have in settings.json, or launch.json, but use some common sense to merge what's above into your existing files.
You can also run the exact same tests from within a Docker container if you want to avoid having to (re)-provision clean installs.
If the code is already present on the host system, you can use the python image, and map the code into the container for the tests:
docker run --rm -it --name sr3-pytest -v $(pwd):/app -w /app python:3 bash
Then you run the Setup section above, starting at Step 3.
If the code isn't already on your system, then the following should get you setup:
- Run container
docker run --rm -it --name sr3-pytest ubuntu bash - Install dependencies (Ubuntu/Debian)
apt-get update && apt-get install -y git python3 python3-pip - Then follow the Setup directions above