Negative testing
Generate rows that break one declared constraint each, then grade what the target actually rejected.
A test that sends only valid data proves the happy path. Negative testing sends rows that each break exactly one constraint you declared, loads them, and then checks two things: that the target refused every one of them, and that it accepted everything valid alongside them.
Use it against a sandbox. These rows are designed to be refused, and a target that accepts them has stored data you did not want it to have.
Declare the constraint on the field
A negative set is built from the template, so the constraints live next to the generator options as options.invalid on the fields under test:
{
"name": "country_code",
"type": "Custom",
"options": {
"values": ["DE", "AT", "CH"],
"invalid": { "kind": "enum", "note": "Fusion rejects unknown country codes" }
}
}kind says which declared constraint the mutation violates, and each kind produces one violating value:
| Kind | The violating value |
|---|---|
required | An empty value. |
type | An array, whatever the field's type was. |
length | The value padded out to one character longer than the field's maxLength, falling back to max, then to 255 when the field declares neither. A value already that long comes back unchanged, so it is not a violation. |
format | A value that cannot parse as the field's format. |
enum | A value outside the field's configured list. |
range | One more than the field's max, or Number.MAX_SAFE_INTEGER when max cannot be read as a number. |
note is optional and is the tester's own: it travels with the expectation and comes back on the finding, so a failure names the rule someone had in mind rather than only the field.
Two rules are enforced when the set is built. A field carrying an options.invalid that cannot be parsed fails the build rather than being quietly skipped, and the key field may not carry one at all - a mutated key cannot attribute its own rejection.
Build the set
POST /negative-sets, or generate_negative_set from the agent, takes the template, a ready dataset of valid rows to mutate, the key field, and how many mutations to make per field (1 to 50, default 1).
What comes back is a sealed dataset containing every valid row plus, for each field carrying a spec, rows identical to a valid row except for that one violation. Each mutated row gets a derived unique key, and the expectation set records that key with the field, the violation kind and the outcome expected of it.
One violation per row is the invariant the whole workflow rests on. A row with two defects cannot attribute a rejection: the target refuses it, and you cannot say which constraint did the refusing.
Load it, then grade it
Load the negative dataset the way you would load any other - it is an ordinary dataset and needs the same approval - and let the rejections land in the job's quarantine.
Then POST /negative-runs/grade, or grade_negative_run, with the load job's id and the negative dataset's id. For a file load, pass either the file load's id or the poll it handed off to: a file load quarantines nothing itself, so the grade reads the quarantine of the poll that ingested the target's result, and that poll needs a result block to record any rejections. The job must have delivered the negative dataset, directly or through a transform or mask derived from it, and both must be in your project. It compares the expectation set against the keys the target actually rejected and reports findings in both directions:
| Finding | What it means |
|---|---|
accepted_mutation | The target took a row that breaks a constraint you declared. Either its validation has a gap, or the constraint is not what you believed it was. |
rejected_valid_row | The target refused a row that was valid. A defect in the data, the mapping or the interface - and one a positive-only run would have shown as an unexplained rejection. |
The summary counts the mutations sent, how many were rejected and accepted, the valid rows sent, and how many of those were rejected. A run is clean only when every mutation was rejected and every valid row was accepted.
A grade is refused rather than computed from a quarantine that cannot hold the answer, because a mutation missing from the quarantine counts as accepted. The route answers 409 until the job, or the poll a file load handed off to, has completed (a failed or cancelled run is refused too), and 400 for a job that did not deliver the negative dataset or records no rejections, such as a mask or a poll without a result block.
Read the result honestly
A clean grade says the target enforced the constraints you declared, on the rows you sent. It says nothing about constraints you did not declare, and nothing about fields with no options.invalid spec on them.
The grade is computed from the quarantine, so it sees the rejections the load recorded. A row that failed to arrive without producing a quarantine entry reads as accepted, which is a finding pointing at the delivery rather than at the target's validation. Check the job's counters as well as the grade.
Quarantine entries carry the record that was rejected, so a negative run's quarantine contains your mutated data and, alongside it, any valid rows the target refused. Keep it restricted like any other quarantine output.
See Field types for the options a constraint is declared against, Data jobs for the load itself, and Reconciliation for the evidence a run leaves behind.