We don't like to admit it, but our code is riddled with bugs. Even for very experienced developers, even if we've seen it all before, even on the fifth time we've written an essentially identical function, our code still has bugs. In this talk we'll look at a library that can automatically uncover these bugs, and learn how to apply it to real-world codebases to find and mediate these problems.
The Hypothesis library by David McIver is a property testing library for Python. Property testing is related to fuzz testing, a technique commonly applied in fields where correctness is paramount. With a little guidance from you, the programmer, Hypothesis can generate a wide variety of valid -- and invalid -- inputs to your functions, and test cases and scenarios you might never have considered. It can generate instances of your custom objects, and so isn't limited just to testing very simple functions. On top of all of this, when Hypothesis finds a failure, it will simplify the failing case, which aids tremendously in diagnosing what is actually wrong in your code.
We'll take a look at what Hypothesis has to offer, and how you can apply it to your codebase. We'll see how to generate custom test case strategies, and how to generate instances of your own objects. Finally, we'll glance under the hood to see how Hypothesis generates test cases, and how it can automatically simplify them, to gain confidence that it's helping us find real bugs and not just flukes.
Dan is an engineer at Dropbox, and organizer of PyGotham in NYC. This is his first time attending North Bay Python.