A modest testing strategy proposal

Principles of Testing

There are some key principles that drive the way I think about testing. It’s important to call these out because it helps you understand why you’re testing in a certain way, and what testing approaches are not as useful as you might think at first…

Enable empowered and independent teams

  • Can make large-scale design changes without significant impact to other teams
  • Can complete work without communicating and coordinating externally
  • Can deploy/release on demand without requiring an integrated test environment
  • Can perform deployments during normal business hours with negligible downtime
  • We can and do deploy our application independent of other applications/services it depends on

Have an economic view of tests

Enable fast detection, root cause analysis, and resolution

Testing Strategy

With these principles in mind, here are some recommendations I would like to offer around testing strategy…

Unit Tests

  • TDD forces you to build only what you need based on the contract you require for the story you are building
  • TDD generally forces you to build a more decomposed design with small, testable units. You can’t build large, complex, and ultimately untestable classes if you’re writing the tests first.

Contract Tests

End-to-End Tests

Where to run end-to-end tests

A common strategy is to build a “staging” environment where everything in the system runs in a state as close to production as possible, including some form of replication. But this strategy has a number of issues, particularly when we keep our testing principles in mind:

  • A staging environment impacts the economic value of tests because it is so difficult to maintain. The more components in your system increases the complexity and failure rate of a staging environment exponentially
  • A staging environment impacts the economic value of tests because staging is not production, so a passed test in staging doesn’t necessarily tell you that the system will succeed in production.
  • A staging environment impacts the economic value of tests because it can be very difficult if not impossible to replicate behavior in staging that you need for certain tests

Use canary deploys to test with the organic traffic in production

  • A mechanism for routing an ever-increasing amount of traffic to the new version of your component
  • Backward-compatibility and forward-compatibility of your component contract so that the old and new can run in tandem
  • A way to very quickly roll back the new version if an error is detected

Acceptance testing

Heartbeat tests of key user journeys

Data validations

Chaos testing

Test data

Should my end-to-end tests use the UI or just be against an API?

  • Build some key tests using your UI to verify your UI is working

Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store