QA and testing

Because it is the only number describing whether quality assurance is working. Tests run measures activity; coverage measures intent; defects found measures what was looked for. Defects that reached production measures what was missed, which is the question.

We also measure which phase the defect belonged to. A requirements defect found in production means the acceptance criterion was wrong, and no amount of testing would have caught it. Classifying by origin is what distinguishes improving tests from improving the process.

And the cost of each escape, because not all are equal. A cosmetic defect and one that corrupts data should not count the same in an aggregate metric, and a team assessed by count learns to prioritise what is easy to find.

Involved before the code exists, in defining acceptance criteria. That is where quality assurance has most value and where it is most frequently absent: a team receiving finished functionality to test can only find implementation defects, never requirements defects.

With access to the same context as the development team: user requests, incident reports and usage metrics. A test team that does not know which functionality is most used tests everything at the same priority, which is the same as having no priority.

And without a test phase at the end. Testing at the end produces a negotiation about what gets fixed before release, with the date already committed, and that negotiation is always lost. Continuous testing produces defects found while fixing them is still cheap.

In Portugal or Poland, inside the European Economic Area, where tests run over data resembling production. It is the model that avoids the international transfer question and what we propose by default for products with European users.

In the global network where test environments run on synthetic data, which is the design we recommend regardless of location. Testing on real user data is a practice that persists and has no reasonable defence.

Time zone matters less here than in almost any other service, because most of the work is asynchronous. What matters is enough overlap for a daily meeting with the development team, and half a day is enough.

By risk rather than by coverage. Code coverage measures which lines were executed and says little about whether what matters works; risk combines likelihood of failure with consequence, and it is what decides where effort is worthwhile.

The highest-risk areas are identifiable and almost always the same: recently changed code, integrations with third-party systems, flows involving money, and functionality that has historically produced incidents. A map of those four dimensions guides better than any percentage.

And we revise that map against real incidents. An area producing three production incidents last quarter deserves attention regardless of what coverage says, and an area stable for two years probably does not need the exhaustive regression that keeps running out of inertia.