Review The Output

English · Русский

A successful command is not the end of the review. Use the artifacts in the bundle to decide whether the synthetic dataset is safe and useful.

This page describes synthetic-generation bundles. The gated mixed-origin transformation workflow has a separate review contract: validation does not make its output synthetic, and action provenance and any required exact-snapshot local preservation receipt must also be checked. See Selective Transformation.

Review Order

  1. Read generation_manifest.json.
  2. Read validation_report.json.
  3. Read business_validation_report.json when business rules were supplied.
  4. Review the effective dataset_spec.yaml or dataset_spec.json.
  5. Inspect a small sample of generated rows.

Manifest Checks

For a valid synthetic dataset, require all of these conditions:

{
  "synthetic": true,
  "source_rows_copied": false,
  "validation_valid": true
}

Also confirm:

  • seed is the value requested by the caller;
  • row_counts match the requested scale;
  • output_format is correct;
  • package_version and dataset_spec_schema_version are recorded;
  • spec_sha256 is present;
  • business_validation.rules_sha256 and its summary are present when rules were used.
  • effective_rules binds generation mode, validation settings, relationship, constraint, and distribution counts to the exact spec and business-rule fingerprints without repeating rule values.

Validation Report

validation_report.json reports deterministic checks for:

  • entity and field schema;
  • primary-key uniqueness;
  • relationships and foreign keys;
  • field and row constraints;
  • temporal and formula constraints;
  • aggregate reconciliation.

Do not treat a missing report as success. A failed valid-mode run is not an accepted dataset. Explicit mixed/negative test generation may retain intentionally invalid rows and their failed validation report in a supported output format; the CLI reports validation_failed and exit 1. Do not relabel that result valid. Parquet negative datasets use explicitly marked scalar columns when malformed cells cannot fit native Arrow types. The bundle reader restores those cells for validation; their report remains invalid. Ordinary incompatible typed values still reject the entire export without partial publication. Keep the specification, manifest and encoding metadata described in the source/output matrix. Privacy failures never authorize publication of source values.

These synthetic manifest checks do not apply to the separately gated selective transformation workflow. Its bundle records mixed origin, action provenance and any exact-input preservation authority; it must not claim fully synthetic or anonymous output. Transformation validation failure prevents publication. See the execution contract.

Assurance Levels

Interpret each result at its actual level:

Level Evidence What it does not prove
Structural validation Schema, keys, relationships, constraints, formulas, and business rules passed deterministic checks. Values are private or safe to distribute.
Built-in safety policy The reviewed spec rejected known raw sensitive values, unsafe categories, and disabled safety settings. The report's privacy section records this policy check. Detectors found every PII, secret, quasi-identifier, or rare free-text value.
Complete-row reuse check A supported CSV workflow compared generated and available source rows and found no complete-row match. No individual value, partial row, rare combination, or statistical pattern resembles the source.
Domain privacy review An accountable reviewer assessed the generated artifacts under the organization's policy and intended use. A formal statistical privacy guarantee unless a separate qualified evaluation provides one.

validation_valid: true means the enabled deterministic checks passed. It is not a privacy certificate. Name- and value-based detection is heuristic and can produce false negatives, especially for domain identifiers, quasi-identifiers, rare categories, and free text. This project does not assign an anonymity, differential-privacy, or re-identification-resistance level.

Profile Review

Profiles should contain metadata such as types, null ratios, ranges, percentiles, approximate distinct counts, safe low-cardinality distributions, and masked patterns.

Stop and investigate if a profile contains:

  • a real email address, phone number, token, credential, or private key;
  • a source row copied as a nested object or list;
  • rare free-text values that could identify a person;
  • an unexpected identifier distribution.

Generated-Row Review

Inspect generated data for domain usefulness, but do not compare or publish raw source rows during that review. For sensitive domains, perform review in the same protected environment as the source data.

Current CSV bundles declare csv_encoding: backslash-null-v1 in their manifest. String cells and headers that could be spreadsheet formulas receive a reversible \F marker; null is \N, empty text stays empty, and literal leading backslashes are doubled. Numeric values, including negative numbers, remain numeric. Keep the manifest for schema-aware bundle decoding; legacy unmarked CSV retains its previous interpretation. Artifact names supplied through public workflows must be one safe filename component; separators and parent-directory components are rejected before any artifact is written. Formula failures expose only fixed local reasons, never expressions, values, or nested exception text.

Re-run with the same spec, rules, package version, and seed when reproducibility is required. A change to any of those inputs can legitimately change output.

See Safety Model for guarantees and limitations.