Programming

We found that flakiness of e2e tests is usually caused by using libraries, frameworks and devops tools that were not designed for being integrated in e2e tests. So we try to get rid of them, or at least wrap them with devops magic. This requires a skilled devops team and buy-in from management.

At some point we were also solving the issue by having dedicated human reviews of e2e failures, it's easy to train a junior QA engineer to have most false positives quickly retried.

But we would never give up on e2e.

You should end to end test. If it’s slow, do it less often. If it’s costly, again do it less often. But this is your last line of defense against bugs.

Now, every service should also unit test, with mocks of all the other services that they call. I assume this is what you mean by pact testing. These should be locally runnable, offline, as the mocks are cheap offline things. If you need data from a database, I’d recommend mocking the data too.

A pact was broken when a mock passes but an e2e fails. You can share the mock across multiple codebases and test it is accurate to any PRs in the codebase it mocks.

I have never actually managed to get it integrated, but I have always pushed for contract testing that's maintained by both sides of the contract. This has the benefit of being able to create compatibility matrixes between services at the very least

Contract testing and end to end testing should both be different factors that comprise a successful testing strategy. A