AutomatorsDocs
Data and plans

Reconciliation and rejected rows

Check delivery outcomes and review evidence beyond a completed status.

Reconciliation compares expected data with the evidence from generation, transformation and delivery. Use it after a small trial and again after a full run.

Review the run first

Collect the plan or mapping, input and output dataset IDs, job IDs, key-map name and target. Inspect processed, failed and skipped counts. A completed load can still contain rejected rows.

Transformation and load failures can produce an error dataset with the record, error and source position. Treat that dataset as sensitive: it can contain original values. Fix the cause, review the corrected records and use a supported retry or follow-up load. Do not export a quarantine file to a broad audience for convenience.

Masking jobs differ: unmaskable values stop the job rather than being persisted into a raw-data quarantine output.

Create and inspect a report

Use the agent's reconciliation tools or POST /plans/{id}/reconciliation-report with the required inputs from your installed API reference. Reconciliation is a report operation, not another queued data-job kind.

Review what the report actually measured: counts, key coverage, checks and any mismatches or missing evidence. A target accepted response alone may not establish that later business processing succeeded. Follow up on target-side validation results where the workflow requires them.

Count what you expected, and check what arrived

Two routes turn "the job completed" into evidence about the data itself.

POST /datasets/{id}/derive-expectations reads the expectation out of the dataset rather than having someone type it in. It scans every chunk - exact, not sampled - and returns the row count and, when you name a numeric field, a control sum to reconcile against. Store that on the plan's migration coverage item. A countMatchesMetadata of false means the dataset disagrees with its own metadata, which is a partial or corrupt dataset and not a number to reconcile against. The dataset has to be ready; a building one answers 409.

POST /datasets/{id}/sample-verify checks content instead of counts. It samples rows of the mapped dataset, translates each source key through the key map, fetches the record the target actually holds and compares it field by field. It is read-only against the target. Each sampled row comes back as matched, mismatched, unmapped or fetch_failed, and a mismatch reports both values, so a field silently truncated or defaulted by the target is visible without opening the target yourself.

Read the two verdicts that are not about your data carefully. unmapped means the key map has no entry for that source key - the row may never have been delivered. fetch_failed means the check could not reach the record, which is not evidence that it is absent.

Retain enough evidence

Save the report alongside the reviewed plan, run identifiers and signoff where available. Record remaining exceptions and how they were resolved. Avoid declaring success solely because a plan says completed or a job has no top-level error.

To check the other direction - that the target refuses what it should refuse - see Negative testing.

See Plans, Audit and Key maps.

On this page