Using and Setting up a Testing Framework

Setting up Pytest

I decided on using pytest as my test runner because it seemed to be the testrunner that was the simplest to use and put the least amount of work on the developer when writing tests. It didnt require the need to create classes to run tests or the need to designate an entry point when running from command line. To install pytest I just had to use the python package installer using the command pip install pytest. After reading some the pytest documentation and reading this article I felt like I was ready to begin writing my first tests.  

Writing My First Tests

I decided the first thing I wanted to test with the constructer for my TextFile class. To do this I looked at my class and tried to write a test for as many possibilities. This included writing a test to see if the expected result for a nonexistant file, a nonreadable file and a regular valid file all matched. Next I decided to run tests to make sure that the the values stored in the TextFile class were what was expected when a valid file was used to create the object. Writing these tests were useful as it forced me to think about my code and to make sure every pathway was free of bugs. It was also useful because I was able to learn about how you pytest assert statements work as well as how I can capture system output in the test runner as well as how to force an exception. Once I was done writing the tests, I ran the command pytest in my repo directory and was able to see that the tests passed. 

Network Tests

The next challenge was to rest tests to see how my code handled network results. The problem that immediately arises with that is you cant really control what a request is going to return. Fortunately this is a common test that needs to be run so tools to perform network tests already exist and they would I decided to use for my network tests was requests-mock which is already built into pytest. This allowed me to mock what a request to url would return and then write the test to see that my code handled that return the way that was intended. Overall once I was able to see how requests-mock works I was able to check that my code was handling 404, 200 and other status codes correctly. 

Code Analysis and Test Coverage

To see how much my of my code the tests I created were covering I used pytest-cov. This tool allowed me to use the --cov option with pytest and showed what percentage of my code was being covered. When I first ran it I saw that the FileReader file that all of my tests were used to test only had a lower coverage than I expected. So I looked over my code and added tests that were relevant to what I was trying to test and was able to increase the percentage that the coverage report produced by pytest was showing me. 

Continuous Integration 

I added a github actions workflow to project that would run these tests any time a push or pull request was made to my master branch. I was able to do this by adding a python-package.yml to a .github/workflow folder and editing that file to match what I wanted to do for my program. The template that was provided had almost everything I needed to make the actions workflow work the way I intended, really the only thing I had to edit was the python versions to only include python 3.6+ as my code formatter black is not supported below python 3.6 

Final Thoughts 

This introduction to test runners and testing environments was really eye-opening and going forward having tests set up is going to be valuable. This more automated method definitely is more efficient then the manual tests I would run by actually running my code with different inputs and checking myself if the results are what I expected them to be. This became tedious when I had to keep repeating them whenever I changed some functionality or added something new.