QA test data for automated testing

Stable data, actionable failures

Random data can broaden coverage, but an unrepeatable failure is expensive. Give every committed fixture a descriptive seed and export its schema. When a test fails, teammates can regenerate the same rows locally; when the contract changes, update the schema and seed deliberately.

Cover data classes explicitly

• Ordinary values: realistic names, addresses, prices and timestamps for the happy path.
• Missing values: Blank % produces nullable fields at a controlled rate.
• Distinct identifiers: Unique reduces accidental collisions within the generated dataset.
• Domain states: Custom List emits accepted statuses randomly, sequentially or with weights.
• Boundary shapes: Pattern and Regex Pattern create exact identifier formats.
• Time behavior: sequential dates create ordered time-series fixtures.

Match the test layer to a format

Use JSON for browser and unit-test fixtures, SQL or CSV for database setup, NDJSON for ingestion tests and XML for integration contracts. All formats share the same schema and seed.

Playwright fixture example

import users from './fixtures/users.json';

for (const user of users) {
  test(`profile renders for ${user.id}`, async ({ page }) => {
    await page.route('**/api/profile', route => route.fulfill({ json: user }));
    await page.goto('/profile');
  });
}

Seeding strategy for a suite

One seed for an entire test suite is the usual first attempt, and it couples every test to every other one: change the schema for a new case and unrelated assertions start failing on different rows. Two rules avoid that.

Give each scenario its own seed, named after the scenario. checkout-empty-cart, profile-long-names and orders-2024-boundary are independent fixtures that happen to share a generator. Regenerating one cannot disturb another, and a failure name tells you which fixture to reproduce.

Regenerate deliberately, not automatically. A fixture rebuilt on every CI run is not a fixture — it is a source of flake. Commit the exported file, and treat regeneration as a reviewable change like any other, with the seed and schema recorded next to it.

tests/fixtures/
  checkout-empty-cart.json     # seed: checkout-empty-cart
  profile-long-names.json      # seed: profile-long-names
  orders-2024-boundary.ndjson  # seed: orders-2024-boundary
  README.md                    # the schema share-links, one per fixture

What generated data cannot test

Being explicit about the boundary is what keeps the fixture honest. Generated rows give you breadth, volume and repeatability. They do not give you:

• Referential integrity. Each column is generated independently, so an order.customer_id will not correspond to a real customer row. Fixtures that need relationships have to be assembled — generate the parents, then generate children whose foreign keys you draw from a custom list of the parent IDs.
• Business-rule consistency. Nothing enforces that shipped_at comes after ordered_at, or that a cancelled order has no delivery date. If a rule matters, assert it rather than assuming the fixture honours it.
• Realistic distributions. Values are drawn evenly, and production data almost never is. Performance work that depends on skew — hot keys, a handful of enormous accounts — needs a shaped fixture, not a uniform one.
• Your exact edge cases. The empty string, the 256th character, the emoji in a surname, the leap-second timestamp. Hand-author those few rows and keep them alongside the generated bulk.

Avoid false confidence

A large random file is not a test plan. Add hand-authored rows for exact length limits, invalid states, duplicate business keys and timezone transitions. Use generated data for breadth and repeatability, then assert the behavior that matters.

QA fixture checklist

• The schema reflects the production contract without copying production records.
• The seed is named and versioned.
• Null, empty, long and international strings are covered.
• IDs are stable and unique where required.
• Dates include boundaries relevant to the feature.
• Generated payment, email and network values remain test-only.

See the methodology for safety details or the mock API guide for response fixtures.

Common questions

How do I make test data reproducible across a CI run?

Give the seed a name and treat it as part of the test, not as a setting. The same seed and schema produce byte-identical output on any machine, so a failure on CI reproduces locally with the same rows rather than with new ones that happen not to fail.

Last updated 6 September 2026