Month 21 — Joy of TDD — Thrill of refactoring with the safety-net of tests
I wanted to rewrite my Quran SRS project from scratch and made some progress in July. But at some point, I hit a roadblock — I did not know enough of Django to write tests first. So, I launched into reading more about Django. I bought a number of books:
- Speed up Django Tests by Adam Johnson — As I was writing more and more tests, I wanted to find the best way to write Django tests. I could not find any resource to teach me proper way to test Django except this book. I have read a sample chapter and I thought that it would help me to learn about testing from a Django contributor. Though the focus of the book is on speeding up tests, I did learn a number of ideas about how to organize my tests better, what is the real difference between using pytest and unittest for Django testing, etc.
- I bought this book sometime back — Django Crash Course or A Wedge of Django — , but I started going through it again to see whether I can get more out of it. https://www.goodreads.com/book/show/57911746-a-wedge-of-django
- I bought this highly-acclaimed book — Two Scoops of Django — which contains a lot of best practices — Given that it has been a year since I have started learning Django, I thought that I was ready for it. https://www.feldroy.com/books/two-scoops-of-django-3-x
As I was traveling, I was reading these books in PDF form and was thinking about how I can apply the concepts. As I was reading more and more, I realized that this route is going to be long.
But there was an urgent need to fix some bugs as well as add some new features to my app. So, I decided to add the tests so that I get 100% coverage first, and then I could start refactoring.
The route to get 100% coverage was not easy, especially I was getting above the 80% — I became aware of the areas that I had least knowledge of in Django. So, it became a learning path for me based on what I need to add tests.
I got a tremendously useful idea from this video — Oracle of testing — the concept of using existing implementation as a reference for the future. Since my SRS algorithm had grown complex over time, I wrote a fixture that would generate a static snapshot of the algorithm’s final output for all the pages in the system and wrote it to a YAML file. Then, I use the YAML file as a fixture that would act as the Oracle. This made it very easy for me to see what changes would break the algorithm. Initially, it was a bit scary as even unexpected changes broke the algorithm. But as I wrote more and more tests and did more and more refactoring, testing against the Oracle became such a safety net for me. I am so grateful to Justin Crown as well as the Hypothesis testing library team as they were the sources from which I learnt this idea.
I watched a number of videos on pytest — the best of them was a series by Mark Vousden — this is the best intro to pytest in general and specifically to fixtures — takes a very simple example and builds up complexity gradually. His explanation of mocking and parallel testing in video no.3 is also great.
Here is another video by the creator of pytest — I really liked his explanation of when fixtures are useful — to test using different browser.
As I started using pytest more and more, I started customizing it to suit my taste. I wanted to have the BDD style describe-it syntax which is the default in the JS world for testing. So, I found this extension.
Since pytest enables me to write my tests without using the Java-style class-based inheritance which is the default in unittest, I needed some plugin to enable me to get access to the properties that are exposed in those superclasses of Django. So, pytest-django came to the rescue. I really love the default fixtures that this plugin make available like client, db, etc.
https://pytest-django.readthedocs.io/en/latest/
When I was introduced to Jest in JS world, I really liked the watch feature — so, I found pytest-watch — Local continuous test runner with pytest and watchdog.
https://github.com/joeyespo/pytest-watch
So, the tests run automatically as I save any changes I have made to either the tests or to the code and I see the impact — I am really grateful for this wonderful concept of both TDD and as well as refactoring backed up by tests. Despite all the stress associated with changing old code-base, it ended up being a thrill-ride in an adventure park :)