Sets and datasets
Choose inline sets or chunked datasets and move between them.
A set is a small saved row collection you can inspect and reuse. A dataset stores data in chunks for larger processing jobs.
Save a set
From a generation preview or supported scenario result, choose the save-as-set action, name the set and review the rows. The result must be a non-empty array of row objects. A generation API response has column-oriented live_data; convert it to rows first.
Sets are capped at 10,000 rows and 5 MiB of serialized JSON. Oversized results are rejected rather than silently truncated. Use a dataset for larger data.
Lock a reviewed set when it should remain a stable regression fixture. A locked set rejects content, name and description changes and deletion until it is unlocked. Locking a set does not lock the templates or external systems that produced it.
Work with a dataset
Datasets have a lifecycle: building, ready or failed. A producer appends chunks while building, then seals the result before downstream processing. Inspect readiness before starting a transform, mask or load.
The API provides preview, sample and statistics routes under /datasets/{id}. Previews and samples are bounded; a sample is not necessarily a uniform sample of the full dataset, and statistics can report partial coverage. Check the returned coverage or truncation indicators before making claims about the whole source.
Convert between them
POST /sets/{id}/to-datasetconverts a row-shaped set into a ready chunked dataset.POST /datasets/{id}/to-setconverts a ready dataset only when it fits the inline set limits.
The destination's write permission is required. Converting to a set reads the rows into memory, so it is not a route for bypassing set size limits.
Deliver or process data
The interactive export flow offers CSV, JSON, API and database targets for supported row collections. For longer pipelines, use data jobs and inspect their run counters and error outputs before considering delivery complete.
Use a set as FlowProof inputs
Ask chat to prepare FlowProof inputs with their sources and operation checks. Open the saved Set's Automation inputs section: it shows each variable, its effective value, where it came from, and any test-case override. Choose the test case and operation you intend to run.
Values present means the data was found. Evidence passed means saved checks cover those values for that operation within their validity window. Recorded executions are previous results. These are separate: a previous invoice may have passed while today's posting period is closed.
When the selected operation has complete current evidence, Download FlowProof inputs provides inputs.values.yaml, provenance.json and instructions for an operator-controlled handoff. Flowproof 0.12.1 has no --vars flag, so the file is a handoff rather than something Flowproof loads. Credentials remain in the connection or runner environment.
To check the saved values inside DataMaker, choose the case, the operation and the connection to check against, then select Run validation. DataMaker runs a reviewed, read-only Flowproof 0.12.1 flow against that connection with exactly those values and the connection's stored credentials, shows its progress, and lets you cancel it. Only a complete report where every step passed counts as Passed. A wrong value is Failed and lists the failed steps. A run that is cancelled, times out or cannot reach the target is never shown as a pass. The result and its evidence appear in Validation history. Running a validation needs access to the Set, permission to read connections and to run data jobs, and a deployment with the validation worker.
The SAP sales-order check reads one order from an OData V2 sales-order service (such as API_SALES_ORDER_SRV) saved as an endpoint under your SAP connection. It compares the document id, sold-to party and total net amount as exact text: 0000004711 is not 4711, and 1250.00 is not 1250.0. Save expected values exactly as SAP returns them.
Missing values, undocumented sources and stale checks remain visible and prevent export and validation. Ask chat to resolve them through the connected system and save new evidence. For a locked Set, save a new candidate and review the changes. Exporting inputs does not run the test or recheck SAP automatically.