AI-Built API Collections and Scripts: Postman, Bruno, and the Checks You Repeat

Generate the collection from the contract: an OpenAPI file in, Bruno .bru files or Postman collection JSON out, with assertions on status, schema, every business-rule field, and the exact error code on negatives. Get the four scripts analysts always need (token refresh, chaining, polling, signing) in one prompt each. Then put the suite behind a CI gate, which is where it stops being documentation and starts being a control.

In part nine you built a test plan with a hundred and forty cases and a traceability matrix. Thirty of those cases are API cases, and right now they are rows in a table. A row in a table has never caught a defect.

This is part ten of The AI Analyst: turning cases into something that runs, and then into something that blocks a bad merge.

Why is collection building the best AI task an analyst has?

Because every artifact involved is structured text, and structured text is where these models are strongest and easiest to verify.

An OpenAPI contract is YAML. A Bruno request is a .bru file. A Postman collection is JSON. A test script is JavaScript. The transformation from one to another is mechanical, tedious, and completely specified, which is the exact profile of work you should hand over. It is also instantly verifiable: you run it, and it either works against the sandbox or it does not. Compare that to a generated specification, where being wrong is invisible for six weeks.

If you have never built a collection by hand, do that first. Your first API collection in Bruno and Postman takes an hour and it is what lets you tell a good generated collection from a plausible one. Generating things you cannot read is how you end up with a hundred green tests that assert nothing.

Step 1: contract in, collection out

Attached: openapi.yaml for the Merchant Payments API.

Generate a Bruno collection as .bru files, one file per request,
organised in folders by resource.

For every operation in the contract, create:
- the happy path request with a valid body built from the schema
  examples
- one request per documented error response

For every request include assertions:
- res.status equals the expected code
- the response body matches the schema for that response (assert the
  required fields exist and have the right type)
- for success responses, assert the specific value of every field
  that carries a business rule
- for error responses, assert the exact error code and that the
  error object has the documented structure

Use environment variables for the base URL and the token: {{baseUrl}}
and {{token}}. No hardcoded secrets anywhere.

Output the .bru file contents and the folder structure.

A .bru file comes back looking like this, and the important thing is that you can read it:

meta {
  name: Create refund - happy path
  type: http
  seq: 1
}

post {
  url: {{baseUrl}}/payments/{{paymentId}}/refunds
  body: json
  auth: bearer
}

body:json {
  {
    "amount": { "value": 1250, "currency": "EUR" },
    "reason": "CUSTOMER_REQUEST",
    "idempotencyKey": "{{idempotencyKey}}"
  }
}

assert {
  res.status: eq 201
  res.body.status: eq PENDING
  res.body.refundId: isDefined
  res.body.amount.currency: eq EUR
}

Then you run it against the sandbox and fix what breaks. Something always breaks, usually because the contract’s examples are stale, and that is itself a finding worth raising against the contract.

Bruno has a real advantage here: plain files in a folder you own. The assistant writes them, git tracks every change, and a review is an ordinary pull request. Postman works too, and its collection v2.1 JSON generates fine, but you are then reviewing a large JSON blob rather than readable files. Bruno vs Postman for analysts covers the full trade-off; for AI-generated suites in a regulated environment, files in git usually win.

Step 2: assertions that actually prove something

This is where generated collections are usually worthless, so it is where you spend your review.

A generated test that checks res.status: eq 200 proves the server is awake. It passes when the amount is wrong, when the currency is wrong, when the status is wrong, and when the response is a completely different object. Teams then run four hundred of these, see all green, and believe they have coverage.

Four layers, and demand all four:

Layer	What it catches	Example
Status	The endpoint responded as documented	res.status: eq 201
Shape	Fields added, removed, or retyped	Required fields present, types correct
Business fields	The rule is implemented	res.body.status: eq PENDING, res.body.amount.value: eq 1250
Error contract	The right failure, not just a failure	res.status: eq 422 and res.body.code: eq INSUFFICIENT_BALANCE

The error layer is the one people omit and the one that matters most in payments. An endpoint returning 500 where it should return 422 is a defect a customer feels as a broken page instead of a clear message, and a test that only asserts “not 2xx” will never see it. HTTP status codes explained and reason code mapping are the references for getting these right.

Ask explicitly for the schema assertion too:

For each response, generate an assertion that validates the body
against the JSON Schema from the contract's components section,
not just a check that fields exist.

That is what turns your collection into a contract test, catching the day a developer removes a field that a downstream consumer still reads. Contract testing covers why that failure is so expensive and so common.

Step 3: the four scripts analysts always need

These are the things that eat an afternoon each when you write them by hand, and they are all well-specified enough that a model gets them right in one attempt.

Token refresh.

Write a Bruno pre-request script that obtains an OAuth2
client_credentials token and stores it in the runtime variable
`token`, reusing the existing token if it has more than 60 seconds
of life remaining. Token endpoint and credentials come from
environment variables. Handle a non-200 from the token endpoint by
failing loudly with the response body.

Chaining.

Write the post-response script for the Create Payment request that
extracts `paymentId` from the response and sets it as a runtime
variable, and fails the request with a clear message if the field
is absent, so the downstream requests do not run against an
undefined id.

Polling an asynchronous status.

The refund is asynchronous. Write a script that polls
GET /refunds/{{refundId}} every 2 seconds until status is SETTLED or
FAILED, with a maximum of 15 attempts, and fails with the last
observed status if the timeout is reached. Do not use a busy wait.

Signing and idempotency.

Write a pre-request script that generates a UUID v4 idempotency key
and sets it as a variable, and computes an HMAC-SHA256 signature over
the request body using the secret in {{signingSecret}}, setting it in
the X-Signature header. Use the crypto library available in the Bruno
script runtime.

Specify the runtime. Bruno and Postman have different script APIs, and a model given “write a Postman script” for a Bruno collection produces something that looks right and fails at run time. Chaining API requests with JavaScript in Bruno and Postman has the hand-written versions of all four, which is what you check the generated ones against.

The polling script is the one to read most carefully. A generated poller with no attempt cap will hang your CI job for the full timeout, and you will not find out until a build takes forty minutes.

Step 4: the data-driven layer

Your test plan produced a data set. A collection runs one request per file. The bridge is a data file and a single parameterised request.

Attached: the test data CSV from the test plan, and the refund
request .bru file.

Convert the request to use CSV columns as variables, and write the
assertion block so that the expected result comes from the CSV too:
each row carries expectedStatus and expectedErrorCode.

Then give me the bru run command that executes it against the CSV.

Thirty negative cases become one request file and thirty rows. Adding a case is a line in a spreadsheet, which means the testers who do not write scripts can extend the suite, which is the difference between a suite that grows and one that decays. This is the same technique as how to write API test cases, and the data comes straight out of step five of part nine.

Step 5: the CI gate

A collection on your laptop is a personal productivity tool. A collection in the pipeline is a release control, and that distinction is the whole reason to do any of this.

Write a GitHub Actions workflow that runs this Bruno collection on
every pull request to main.

- install the Bruno CLI
- run the collection against the sandbox environment
- take the base URL from a repository variable and the client
  credentials from repository secrets
- run the data-driven negative suite from the CSV as a second step
- publish the results as a job summary
- fail the job if any request fails

Two rules of engagement.

Secrets from the CI secret store, never from a committed file. The generated workflow will do the right thing if you ask; it will also happily inline a placeholder token if you do not, and placeholder tokens get replaced with real ones by someone in a hurry.

Start non-blocking. Run it as an advisory check for a fortnight, fix the flakes, then make it required. A suite that blocks merges while it is still flaky gets disabled within a week and never comes back. Running API tests in CI covers the full setup, including Newman for Postman collections and how to handle environment-specific failures.

What to check before you trust any of it

Generated suites fail in five predictable ways. Fifteen minutes of review catches all of them.

1. Assertions that assert nothing. Search the collection for requests whose only assertion is a status code. Every one is a gap.
2. Invented fields. The model filled a required field from the schema with a plausible value that your business rules reject. It fails on first run, which is fine, but check the fix rather than deleting the assertion.
3. Stale contract examples. The examples in the OpenAPI file were written at design time and no longer match. Raise it against the contract; that is a real defect in your documentation.
4. Hardcoded anything. Grep for http, Bearer, and any identifier that looks real. Secrets in a committed collection are the classic version of this failure.
5. Tests that pass for the wrong reason. The highest-value check: break the thing on purpose. Change an expected value, confirm the test fails, change it back. A suite you have never seen fail is a suite you have no evidence works.

That last one is not optional. I have seen a collection of two hundred requests run green against a completely stopped service, because every assertion was on a field that came back undefined and the check was for presence of the key rather than the value.

The full method for deriving, asserting, and automating API tests, with the banking examples behind these patterns, is in API Testing and QA Mastery for BAs, and the event-driven equivalent is Automate Kafka Validation with Postman.

Beyond collections: the checks you repeat

The same approach covers everything else you do more than twice.

• A reconciliation check between two systems, as a Python script run daily. See reconciliation design for what it must compare.
• A log triage script that pulls one identifier’s trail across services, which is reading production logs made repeatable.
• An event validation harness for Kafka topics, per how to test Kafka.
• A daily data quality check over the tables you keep finding problems in.

The rule for all of them, from automating the analyst workflow: if you can write the rule down completely, it is a script, not a prompt. A script gives the same answer every time, and for a check that gates a decision, determinism is the requirement. Use AI to write the script, then run the script.

The takeaway

Generate collections from the contract, because contract to collection is a mechanical transformation over structured text and it is instantly verifiable. Demand four layers of assertion (status, schema, business fields, exact error code), get the four standard scripts in one prompt each, parameterise the negatives from your test data file, and put the suite behind a CI gate that starts advisory and becomes required.

Then break something on purpose and confirm the suite notices. That is the only evidence that any of it works.

Next: part eleven, where the same approach gets pointed at SQL, the weekly delivery pack, and the dashboard nobody has had time to build. The full path is on The AI Analyst.

Ahmed is a Senior Technical Business Analyst with 10+ years in banking and payments. He builds practical guides and tools for analysts at The Tech BA Toolkit.

Tags: QA, Artificial Intelligence, API Testing, Automation, CI