Locked learning resources

Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Locked learning resources

This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Adding Unit Tests

After setting up the project, this lesson shows you how to write unit tests using pytest. Additionally, flake8 and pytest-cov are installed for further code explorations.

00:00 Now that the project is set up, it’s time to add tests to make sure that we don’t lose functionality as we add code. This will be done in two steps: linting, to look for potential errors and enforced style, and then unit tests. For linting, you’ll use flake8, which combines error and PEP 8 style checks.

00:18 pytest will be used to perform unit tests. To ensure that no tests are missed, you will also use pytest-cov, which calculates how much of the code is covered by unit tests. Go ahead and install these using pip, so pip install flake8 pytest pytest-cov.

00:43 Because you’ve added dependencies to your project, you should go ahead and save these in a requirements.txt file. You can do this by saying pip freeze

00:53 and then a greater than symbol (>) to send the standard output to requirements.txt. Once you run that, you’ll have a requirements.txt file pop up in your directory, and this will list all the packages required to duplicate your environment.

01:12 Okay. If I go back to calculator.py, to run your linter you can just type in flake8 and then add a --statistics option.

01:28 This will check your code for style issues. You can see it found two, where it wanted to have an extra blank line, and then the --statistics flag sums up everything down here.

01:38 So if you have quite a few issues that are found, then you’ll get a nice summary down at the bottom. This should be pretty quick to fix, so just go up to calculator.py and add an extra line in between each of these functions. Now if you rerun that, you’ll get nothing there. All right! Now it’s time to write unit tests.

01:58 Back in your repo, go ahead and make a new file and call this test_calculator.py. The test_ prefix is important because this is how pytest will know that this file contains unit tests. Go ahead and copy the following code,

02:20 and you should be all set. This will import calculator, and then to test it, it’s going to test addition and test subtraction by asserting that you get the right value out of the different methods that you’ve added.

02:33 There are plenty of other tests that you could add, but this is good for now. All right, to run the tests just call pytest -v --cov and add a period (.) at the end to make sure that it only checks the CalculatorLibrary/ directory. If you end up with a ton of output here, it’s probably looking through the virtual environment folder called calculator/.

02:56 Let’s run this, and I’m just going to bring up the terminal here a little bit more to read it better. The -v makes it a little more verbose and you can see which tests passed, which ones failed. And then the --cov option shows the coverage, so you can see that 100% of calculator.py is checked. Awesome!

03:17 You can see everything passed. And if you look, you should have your .gitignore, your calculator, your README, a requirements.txt, and a test_calculator.py file.

03:28 There’s going to be extra cache folders and your .git/ folder, but your .gitignore should check for that. So, if you check your status, you can see that you modified calculator to fix the style, and that you’ve added your requirements and your test_calculator files.

03:45 So let’s just git add all of those,

03:51 and then commit them. And you can say 'Add unit tests for calculator'. All right, and to get that back onto GitHub just do a git push.

04:08 And there you go! Now that you’ve defined your dependencies in the requirements.txt file and added your unit tests, it’s time to connect this project to CircleCI to set up your CI pipeline. Thanks for watching.

Avatar image for adoormouse

adoormouse on Sept. 10, 2019

Code coverage tests run into python 3 site packages. Still looking for a good resolution to this, see below:

(venv_calculator)$ pytest -v --cov 
calculator.py                                                                                                  4      0   100%
test_calculator.py                                                                                             6      0   100%
/home/home_user/src/RealPython/venv_calculator/lib/python3.5/site-packages/_pytest/_argcomplete.py              34     33     3%
........
TOTAL                                           15185  12523    18%

Possible solution, however:

$ pytest -v --cov=test_calculator.py
Module test_calculator.py was never imported. (module-not-imported)
Coverage.py warning: No data was collected. (no-data-collected)
WARNING: Failed to generate report: No data to report.
Avatar image for adoormouse

adoormouse on Sept. 10, 2019

Doc at the following location needs to be updated: realpython.com/python-continuous-integration/

The following command runs your test:
$ pytest -v --cov

This vid is correct in using

$ pytest -v --cov .
Avatar image for Mahidhar Nyayapati

Mahidhar Nyayapati on Oct. 20, 2019

Why do we need to install and use flake8. Cant we just avoid this step and directly go to unit-testing with py-test.

I have a project code that runs smoothly without any programmatic or logical errors. But flake8 says “trailing whitrespace”,”extra lines”, “extra space in new line”, “line length too long”. These errors are more based on the linting or in other words making the program more “beautiful in looks” which is not necessary.

Currently the code is more than 1500 lines of code with all variable names already specified and cannot be changed.

By avoiding flake8 and directly going to pytest these flake8 errors do appear in CircleCI builds.

Please tell me what should I do in this situation.

Avatar image for Joe Tatusko

Joe Tatusko RP Team on Oct. 21, 2019

Hi Mahidhar, flake8 isn’t required but helps your code conform to PEP 8 standards. While not necessary, it does help keep your code more readable for others who may be working on it. Larger projects in particular benefit from strict style guides, as they can quickly become unmanageable if things aren’t kept consistent.

Based on the type of linting you are seeing from flake8, I recommend taking a look at black which is another style-focused tool. Instead of telling you where there are problems, however, it will automatically make fixes like removing whitespace and adding extra lines. Try making a copy of your project (since it will automatically make changes) and seeing if that helps you out!

Avatar image for W Patrick Jones

W Patrick Jones on Dec. 15, 2019

I just ran flake8 for the first time as per the tutorial and it ran against every file in my venv. Including every site-package. Can you add info on how to tell flake8 to only evaluate certain folders/files?

Avatar image for wsherby

wsherby on Feb. 29, 2020

I have the same problem as Patrick, where flake8 runs against every file in the venv if I don’t specify the python file I want to check. How did you set this up to not evaluate venv?

Avatar image for wsherby

wsherby on Feb. 29, 2020

@Patrick – I had this problem and I just figured it out. He has his venv directory outside of his library directory. This way you run the linter on the library only.

Avatar image for Billa123

Billa123 on March 24, 2020

I’m seeing bunch of errors with this command: pytest -v --cov=testcalculator.py

=================================================================== short test summary info ===================================================================
ERROR lib/python3.7/site-packages/jsonschema/tests/test_jsonschema_test_suite.py - ValueError: Can't find the JSON-Schema-Test-Suite directory. Set the 'JSO...
ERROR lib/python3.7/site-packages/zope/interface/common/tests/test_collections.py - AssertionError: ('test_auto_ParseResults_zope_interface_common_collectio...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Avatar image for David Alvarez

David Alvarez on June 18, 2020

About flake8 inspecting venv I was looking for a solution and found this. Not sure is the proper/optimal way to do it. Creat a setup.cfg in root folder of the project (or edit it if exists) with the following data:

[flake8]
exclude = .git, <venv_name> 
max-line-length = 119

where <venv_name> is the name you gave to venv folder

Avatar image for Wesley

Wesley on Aug. 5, 2020

This method did not work for me.

I had to go on SO, where I was told to put an empty conftest.py in my project root, and then run pytest from the project root. This worked. I’d like to know why the method in the video did not work.

Avatar image for Vasanth Baskaran

Vasanth Baskaran on Aug. 25, 2021

(calculator) (base) vasanth@Vasanths-MacBook-Pro calculatorLibrary % pytest -v --cov .
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --cov
  inifile: None
  rootdir: /Users/vasanth/Documents/Projects/python-projects/calculatorLibrary

(calculator) (base) vasanth@Vasanths-MacBook-Pro calculatorLibrary % pytest -v --cov test_calculator.py
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --cov
  inifile: None
  rootdir: /Users/vasanth/Documents/Projects/python-projects/calculatorLibrary

(calculator) (base) vasanth@Vasanths-MacBook-Pro calculatorLibrary % pytest  --cov test_calculator.py 
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --cov
  inifile: None
  rootdir: /Users/vasanth/Documents/Projects/python-projects/calculatorLibrary

(calculator) (base) vasanth@Vasanths-MacBook-Pro calculatorLibrary % pip install pytest-cov            
Requirement already satisfied: pytest-cov in ./calculator/lib/python3.8/site-packages (2.12.1)
Requirement already satisfied: toml in ./calculator/lib/python3.8/site-packages (from pytest-cov) (0.10.2)
Requirement already satisfied: pytest>=4.6 in ./calculator/lib/python3.8/site-packages (from pytest-cov) (6.2.4)
Requirement already satisfied: coverage>=5.2.1 in ./calculator/lib/python3.8/site-packages (from pytest-cov) (5.5)
Requirement already satisfied: attrs>=19.2.0 in ./calculator/lib/python3.8/site-packages (from pytest>=4.6->pytest-cov) (21.2.0)
Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in ./calculator/lib/python3.8/site-packages (from pytest>=4.6->pytest-cov) (0.13.1)
Requirement already satisfied: iniconfig in ./calculator/lib/python3.8/site-packages (from pytest>=4.6->pytest-cov) (1.1.1)
Requirement already satisfied: packaging in ./calculator/lib/python3.8/site-packages (from pytest>=4.6->pytest-cov) (21.0)
Requirement already satisfied: py>=1.8.2 in ./calculator/lib/python3.8/site-packages (from pytest>=4.6->pytest-cov) (1.10.0)
Requirement already satisfied: pyparsing>=2.0.2 in ./calculator/lib/python3.8/site-packages (from packaging->pytest>=4.6->pytest-cov) (2.4.7)
WARNING: You are using pip version 20.2.3; however, version 21.2.4 is available.
You should consider upgrading via the '/Users/vasanth/Documents/Projects/python-projects/calculatorLibrary/calculator/bin/python -m pip install --upgrade pip' command.
(calculator) (base) vasanth@Vasanths-MacBook-Pro calculatorLibrary % pytest -v --cov test_calculator.py
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --cov
  inifile: None
  rootdir: /Users/vasanth/Documents/Projects/python-projects/calculatorLibrary

(calculator) (base) vasanth@Vasanths-MacBook-Pro calculatorLibrary % pytest -v --cov                   
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --cov
  inifile: None
  rootdir: /Users/vasanth/Documents/Projects/python-projects/calculatorLibrary
Avatar image for Vasanth Baskaran

Vasanth Baskaran on Aug. 25, 2021

I am getting the error “unreocgnised arguments” for –cov. I have created the virtual envrionement in the folder where the project files are present and I hope that is not an issue.

Become a Member to join the conversation.