Develop internal training material for python testing

I want to develop internal training content for writing good python unit and integration tests. There are three deliverables for this project: * A longform document describing and exploring the subject matter in depth * A 20 minute presentation walking through the content and examples * An interactive tutorial / workshop applying the concepts in the document to concrete examples, with some automated evaluation based on coverage and hidden implementation bugs. This may be done either as a jupyter notebook, a dockerised training repository, an interactive webpage or similar, but ideally there should not be any backend necessary. Familiarity with the subject matter is required, but I will be available to help develop and edit the content. The examples will walk through a simple microservices stack utilising the following python technologies: * Flask HTTP server * This interprets a REST query, validates the input and then dispatches the query to a microservice via a thrift RPC call * Apache Thrift mid-layer microservice * This implements permission checks and aggregates and interprets data into a series of database requests in one of the two data layers described below * SQLAlchemy query constructed within the same service * Translating a thrift request to a similarly-structured SQLAlchemy query * Via another thrift microservice that uses mysql connector and raw SQL as a DAO service A skeleton describing the content is provided below. It may be incomplete - if there is anything else to cover, please expand it or alter it. Please note that the examples do not have to be written from scratch - some existing skeleton code exists, just focus on the test structure and content. Testing Best practices * Test only public methods/functions * Test only the external behaviour of a class or module, otherwise you are just making the implementation details harder to change * Writing descriptive tests * Each test should have a docstring describing the property being validated * Quick note on TDD * Keep tests DRY and isolated * Test code is code too - share mocks and setup methods and keep it clean * Avoid complex code in tests * Tests need to be obviously correct - if you’re doing a lot of computation in a test, it is too complex. * Data providers * Don’t hard-code data, use data providers. This makes tests generalisable as property tests, too * Golden Tests * Quick note on hypothesis * Avoid mocks & a note on DI * Mocks are fragile - avoid them where you can. Your functions should be as much of a black box to your tests as possible. DI-like patterns help with this. Mocking * Mock at the service boundaries * This tests as much of the real behaviour of your code as possible. * Wrap your service boundaries with an adapter that isolates your code from the interface of the external system. * Python mock library * Basic usage, common pitfalls, examples, etc * How to write good mocks * Unit tests vs Integration tests * What’s the difference? Why would you use either? * When not to test * Type systems and other static tooling * Coverage vs defensiveness * Enforcing Contracts Test runners * Nosetests, pytest * Skipping tests, selecting tests, command line arguments, data providers[d][e][f] * Coverage * If it’s not covered, do you need it? Tools: Pytest watch, pdb (and ptpdb), vscode integrations Testing in the example stack * Flask * creating and using the flask mocks * A simple test * Thrift * Using a local thrift server mock * Pure python * Testing a simple permission check * SQL * Testing using a local mysql instance * SQLAlchemy * Same as above, but with sqlalchemy set up Advanced testing * Hypothesis[g] and Property based testing / state machine testing