Why features pass QA and still break in production

Database migrations are where the mock data problem shows up most clearly. A migration that adds an index to a table with 500 rows in the development database runs in milliseconds and passes every test. The same migration against a production table with 8 million rows locks the table for 90 seconds during peak traffic. Nobody saw it coming because nobody tested it against 8 million rows.

This isn't an edge case. It's a routine failure mode, and it happens because the data used for testing doesn't reflect the data the application actually runs against.

What mock data doesn't tell you

Key takeaway: Mock data and fixtures validate that code handles the expected case. Production data validates that code handles the actual case. The gap between those two things is where most production incidents originate.

Fixtures and factories are useful. They give you a controlled, repeatable test environment. They let you write tests that run in seconds, isolate specific scenarios, and produce consistent results. None of that stops being true.

What they can't give you is the complexity that accumulates in a production database over time.

A production Drupal installation with three years of content has orphaned taxonomy terms, nodes with missing field data from a schema migration that didn't fully complete, content types with character encoding anomalies from an import run in 2021, and user accounts with permission combinations that were possible under an older role configuration but shouldn't exist under the current one. None of that is in the fixtures, because the fixtures were written to represent how the data should look, not how it actually looks after years of real-world use.

A production Django application with active users has query patterns that a development database doesn't. Certain filters that are fast against 10,000 records become full table scans against 2 million. Certain joins that work fine in isolation behave differently under the load patterns that real usage creates. An N+1 query that's imperceptible in development becomes a performance incident in production.

The mock data problem isn't that developers are writing bad fixtures. It's that fixtures are a model of reality, and models are always incomplete.

What "production complexity" actually means

Key takeaway: Production databases are complex in three ways that fixtures can't replicate: volume (the amount of data), entropy (the accumulated inconsistencies of real-world use), and query behavior (how the database engine actually performs under real load patterns). All three matter for testing.

Volume is the most obvious dimension. A feature that handles 100 records handles 10 million records differently, in ways that depend on indexing, query planning, and memory allocation. Testing at development scale doesn't always surface these differences.

Entropy is less visible but equally important. Real production databases contain data that doesn't conform to current schema expectations, because schemas evolve and migrations are rarely perfectly retroactive. A Drupal content migration from five years ago may have left fields in states that the current application code doesn't expect to encounter, because the code was written after the migration and assumes clean data. Testing against fixtures written to match the current schema misses these cases entirely.

Query behavior is the hardest to replicate artificially. Database engines make different decisions about query plans depending on table statistics, index cardinality, and data distribution. A query that runs efficiently against a small, uniform development dataset may trigger a different execution plan against a large, heterogeneous production dataset. The only way to test for this reliably is to test against data that has the same statistical properties as production.

How data cloning works in practice

Key takeaway: Byte-level data cloning creates a production-equivalent copy of the production database state in a preview environment. Combined with data sanitization, this gives developers a test surface with production complexity and no exposure of real user data.

Upsun is a platform-as-a-service that manages the infrastructure layer of your application stack so your team doesn't have to. When a preview environment is created from a production branch on Upsun, the environment includes a byte-level clone of the production database at that point in time. Not a schema copy. Not a fixture-seeded blank database. A faithful copy of the data as it exists in production at clone time, down to the same orphaned records, the same encoding anomalies, the same query statistics.

The clone happens at the storage layer, which means it's fast regardless of database size. An 80GB database doesn't take 80GB worth of time to clone because the platform handles it as a filesystem operation rather than a data transfer.

For most teams, the process looks like this:

# Create a new environment branched from production
upsun branch feature/new-search-index main

# The environment spins up with a full clone of production data
# Your feature code runs against real data from the first commit

The developer's branch environment now has the same 8 million rows the migration will encounter in production. The index addition that locked the production table for 90 seconds shows up in development, before it's deployed, when it can still be fixed.

Sanitization: making production data safe to use

Key takeaway: Production data cloning is only useful if the cloned data can be safely accessed by developers and shared with QA teams. Sanitization strips or masks personally identifiable information before the clone reaches a non-production environment, satisfying both privacy requirements and the practical needs of regulated development workflows.

The obvious concern with bringing production data into development environments is privacy. Production databases contain real user data: email addresses, names, payment references, health records, whatever the application stores. That data shouldn't be accessible to every developer on the team, and it shouldn't appear in preview environment URLs that get shared with clients for review.

Data sanitization runs as part of the branching process, before the cloned data reaches the preview environment. A sanitization hook replaces sensitive fields with realistic but fictional values: real email addresses become generated ones, names are replaced, payment data is masked. The database retains its structural complexity and its volume while the identifying information is gone.

For a Drupal project, a sanitization hook using Drush might look like this:

hooks:
  deploy: |
    if [ "$PLATFORM_ENVIRONMENT_TYPE" != production ]; then
      drush -y sql:sanitize --sanitize-email=user+%uid@example.com
    fi

The hook runs automatically on every non-production environment, which means sanitization is structural rather than procedural. It doesn't rely on someone remembering to run it; it runs because the configuration says it does.

The result is a preview environment with production-scale data, production-complexity query patterns, and no real user information. QA can test against it. Developers can debug against it. Auditors can verify that the environment reflects production conditions. None of them are looking at real user data.

What this changes for QA and release confidence

Key takeaway: When QA signs off on a feature that was tested against real production data, the sign-off means something different than when it was tested against fixtures. The gap between "QA approved" and "behaves in production" closes significantly.

The practical shift is in what test results mean.

When a feature passes QA in a fixture-seeded environment, the result is: this feature works correctly against idealized data in a controlled environment. That's useful, but it leaves open a category of failures that only production data would reveal.

When a feature passes QA in an environment cloned from production, the result is: this feature works correctly against the actual data it will run against in production, including the edge cases, the volume, and the accumulated entropy of real-world use. That's a meaningfully stronger signal.

For regulated applications where QA sign-off is part of a compliance process, this distinction matters to auditors. Evidence of testing against production-equivalent data is stronger evidence than testing against fixtures, because it demonstrates that the test conditions reflected reality.

For teams that have experienced production incidents traced back to data the fixtures didn't cover, it removes a category of uncertainty that currently sits between deployment and confidence.

Already using fixtures and mocks?

Keep them. Fixtures are the right tool for unit tests, isolated scenarios, and fast feedback loops. Production data cloning sits alongside them, not in place of them: fixtures for testing how your code should behave, production data for testing how it actually behaves at scale. The two approaches answer different questions, and a mature testing setup uses both.

Frequently asked questions (FAQs)

Is production data cloning safe from a GDPR or HIPAA perspective?

The clone itself is a copy of production data and carries the same obligations as production until it's sanitized. The sanitization hook runs as part of the environment creation process and should be treated as a required step for any regulated application. Once sanitized, the environment contains no real personal data and can be accessed by developers and QA without additional data handling obligations.

How current is the cloned data?

The clone reflects production data at the point the branch was created. It's a snapshot, not a live sync. For most development and QA purposes this is sufficient, since the goal is production-equivalent complexity rather than real-time data. For testing that specifically requires recent data, branches can be re-created from the latest production snapshot.

Does cloning a large production database slow down environment creation?

Not significantly. The clone happens at the storage layer as a filesystem operation, which means the time taken is largely independent of database size. An environment branched from a production database with 80GB of data takes roughly the same time to create as one with 1GB.

What if our production database contains data we can't bring into any non-production environment, even sanitized?

The sanitization hook can be configured to exclude specific tables entirely rather than masking them. For data that can't leave production under any circumstances, the pattern is to exclude those tables from the clone and seed them with synthetic data instead. The rest of the database retains its production complexity.

Can we run the sanitization hook against specific data types rather than the whole database?

Yes. Sanitization hooks have full access to the database and can apply different treatments to different tables: masking email addresses in one table, replacing names in another, and leaving non-sensitive tables untouched. The configuration is as granular as the application requires.