Anthony Shaw joins Brian to discuss flaky tests and flaky test suites.
- What are flaky tests?
- Is it the same as fragile tests?
- Why are they bad?
- How do we deal with them?
- What causes flakiness?
- How can we fix them?
- How can we avoid them?
- Proactively rooting out flakiness
- Test design
- GUI tests
- Sharing solutions
Get started with a free $100 credit toward your first project on DigitalOcean and experience everything the platform has to offer, such as: cloud firewalls, real-time monitoring and alerts, global datacenters, object storage, and the best support anywhere. Claim your credit today at: do.co/testandcode
- Dropbox article on flaky tests
- Microsoft article on flaky tests
- pytest-rerunfailures: a py.test plugin that re-runs failed tests up to -n times to eliminate flakey failures
- pytest-randomly: Pytest plugin to randomly order tests and control random.seed
- pytest-random-order: pytest plugin to randomise the order of tests with some control over the randomness
- pytest.approx() — approx
- Anthony's testing article on RealPython
- Ghost Inspector
- wily: A Python application for tracking, reporting on timing and complexity in tests