Schemathesis API Fuzzing: Property-Based Tests From Your OpenAPI Contract

Schemathesis is an open-source tool that does property-based API testing from your OpenAPI contract: it generates valid and invalid requests from the schemas, sends them to a running API, and checks every response against rules that must always hold, such as no 5xx errors, documented status codes only, and response bodies that match the contract. One command, schemathesis run, finds the unicode crash, the boundary amount, and the missing validation that hand-written test cases miss. The analyst’s job is triage: deciding whether each finding is a code defect or a contract defect.

The first time I pointed a fuzzer at a payments API, it found a 500 within minutes: a creditor name with characters outside the Latin alphabet broke a field mapping on the way to the core system. Forty hand-written test cases had passed. None of them would ever have contained that string, because none of us would have thought to type it. This article is the method I now use, written for analysts who already write API test cases and want the cases they would never think of. It belongs to the advanced track of APIs for Analysts and to the QA analyst hub.

What is property-based API testing?

A hand-written test states one input and one expected output: POST /payments with amount 100.00 EUR returns 201. A property-based test states a rule that must hold for every input, then lets a generator try to break it. “The API never returns a 5xx.” “Every response matches the schema the contract declares for that status code.” “A request that violates the schema is rejected with a 4xx.”

The idea comes from Hypothesis, the Python property-based testing library that Schemathesis is built on, and it is the same idea covered at unit level in developer testing in the AI era. What changes at the API level is the source of the generator: the OpenAPI contract. Every type, pattern, minimum, maxLength, enum, and required in the schema becomes both a source of valid data and a target for invalid data.

What does Schemathesis check?

Schemathesis 4 (4.30.0 at the time of writing, October 2026) runs a set of built-in checks on every response. All of them are enabled by default except max_response_time.

Check	The property	What a failure usually means
not_a_server_error	No 5xx, ever	Code defect: unhandled input
status_code_conformance	Status code is documented	Contract drift or missing error response
content_type_conformance	Content-Type is documented	Contract drift
response_headers_conformance	Headers match their schemas	Contract drift
response_schema_conformance	Body matches the response schema	Contract drift or a real field defect
negative_data_rejection	Invalid input is rejected	Missing validation
positive_data_acceptance	Valid input is accepted	Schema too loose, or code too strict
missing_required_header	Missing required header is rejected	Header not enforced
unsupported_method	Undocumented methods return 405	Routing exposes more than documented
allow_header_conformance	Allow header matches the methods	Minor contract issue
use_after_free	Deleted resources are gone	State defect
ensure_resource_availability	Created resources can be read back	Async creation not documented
ignored_auth	Declared auth is enforced	Security defect

That last row deserves its own sentence: ignored_auth tells you when an operation declares a security scheme but answers without valid credentials. Treat it as a security finding and route it as API security testing describes.

How do you run Schemathesis against an API?

Install it as a tool (Python 3.10 or later) or run it without installing through uvx:

uv tool install schemathesis       # or: pip install schemathesis
schemathesis --version

Then run it against the contract and a test environment. st is the short alias for schemathesis:

st run openapi.yaml \
  --url https://payments-fuzz.test.internal \
  --header "Authorization: Bearer $FUZZ_TOKEN" \
  --include-path /payments \
  --max-examples 200 \
  --report junit \
  --report-junit-path results/schemathesis.xml

The options worth knowing:

Option	Use
--url	Base URL, required when the schema is a file
-H, --header	Auth and other headers; tokens are masked in output by default
--checks, --exclude-checks	Choose checks, for example --checks not_a_server_error,response_schema_conformance
--mode positive, negative, all	Valid data only, invalid data only, or both (the default is all)
--phases examples,coverage,fuzzing,stateful	Which phases run (all by default)
-n, --max-examples	Cases per operation in fuzzing, sequences in stateful (default 100)
--include-path, --include-method	Scope the run
--continue-on-failure	Keep testing an operation after its first failure
--seed	Reproduce a run
--report junit and --report-junit-path	JUnit XML for CI

Schemathesis runs in four phases. Examples sends the examples documented in the contract. Coverage deterministically targets every constraint: boundary values, pattern-matching strings, and each enum value. Fuzzing generates random data within, and in negative mode outside, the schema. Stateful chains operations together. Every failure prints a minimal curl command to reproduce it, and st replay re-sends saved failing cases after a fix.

What does a payments contract look like to a fuzzer?

Here is the request body for a payment initiation endpoint, simplified from the shape most European payment APIs use:

CreatePayment:
  type: object
  required: [amount, creditorIban, endToEndId]
  properties:
    amount:
      type: object
      required: [value, currency]
      properties:
        value:
          type: string
          pattern: '^\d{1,15}\.\d{2}$'
        currency:
          type: string
          pattern: '^[A-Z]{3}$'
    creditorIban:
      type: string
      format: iban
      maxLength: 34
    creditorName:
      type: string
      maxLength: 70
    endToEndId:
      type: string
      maxLength: 35

The analyst reads that and thinks “amount, currency, IBAN, reference.” Schemathesis reads it and generates, among thousands of others:

• value of "0.00", "999999999999999.99", and a sixteen-digit value one character past the pattern.
• currency values such as "QZV": they match ^[A-Z]{3}$ but are not currencies your scheme supports.
• creditorName of exactly 70 characters and 71 characters, plus, in the fuzzing phase, strings built from non-Latin and combining characters that a database column sized in bytes may not hold.
• endToEndId of 35 characters, the ISO 20022 maximum, and 36.
• Bodies with amount missing, with amount as a number instead of an object, and with an extra unknown field.

That is a negative test design session done by a machine in seconds. It is not smarter than you; it is more patient.

Teach it what an IBAN is. format: iban is not a standard JSON Schema format, so Schemathesis has no built-in generator for it, generated values are almost never valid IBANs, and most positive requests die at IBAN validation before reaching the interesting code. Register a strategy in a hooks file so positive cases carry valid IBANs:

# hooks.py
import schemathesis
from hypothesis import strategies as st

TEST_IBANS = [
    "DE89370400440532013000",
    "GB82WEST12345698765432",
    "FR1420041010050500013M02606",
]

schemathesis.openapi.format("iban", st.sampled_from(TEST_IBANS))

SCHEMATHESIS_HOOKS=hooks.py st run openapi.yaml --url https://payments-fuzz.test.internal

The same hooks file can hold a business-rule check, so the fuzzer asserts what the contract cannot express:

@schemathesis.check
def amount_is_echoed(ctx, response, case):
    if case.method == "POST" and response.status_code == 201:
        if not isinstance(case.body, dict) or "amount" not in case.body:
            return
        sent = case.body["amount"]
        got = response.json()["amount"]
        if got != sent:
            raise AssertionError(f"amount changed: sent {sent}, got {got}")

How does stateful testing work?

Single requests miss defects that only appear in a sequence: create a payment, read it, cancel it, read it again. Schemathesis chains operations through OpenAPI links, and it also infers links where they are not declared, including from Location headers it sees in earlier phases. Declaring them makes the chain explicit:

paths:
  /payments:
    post:
      operationId: createPayment
      responses:
        '201':
          links:
            GetPayment:
              operationId: getPayment
              parameters:
                paymentId: '$response.body#/paymentId'

In the stateful phase, ensure_resource_availability checks that a payment you just created can be read back. On a payments API that often fails for a legitimate reason: creation returns 202 Accepted and the payment appears a few hundred milliseconds later. That is not a code defect. It is an undocumented eventual-consistency behaviour, and the fix is in the contract: document the 202 and the polling expectation.

How should an analyst triage Schemathesis findings?

This is where the analyst earns their place. A fuzzer produces findings; it does not produce decisions. Every finding goes into one of these buckets:

Finding	Ask	Usually becomes
Server error (5xx)	Could any input justify a crash?	Code defect, always. Raise with the curl reproducer
Undocumented status code	Is the code right and the contract silent?	Contract fix: document the response
Response violates schema	Which is intended, the field or the schema?	Contract fix if the schema is stale; code defect if a consumer depends on the schema
Invalid data accepted	Should this input ever be accepted?	Code defect: missing validation
Valid data rejected	Is the schema looser than the business rule?	Contract fix: tighten with enum, minimum, or a pattern
Declared auth not enforced	Is this operation meant to be public?	Security defect, high priority
Created resource not readable	Is creation asynchronous by design?	Contract fix: document 202 and polling

Two examples from the payment contract above. Schemathesis sends currency: "QZV" and gets 400: positive_data_acceptance fails, because the schema says any three uppercase letters are valid. The code is right; the contract is too loose. The fix is enum: [EUR] for a SEPA endpoint. Schemathesis sends currency: "eur" and gets 201: negative_data_rejection fails, because the pattern requires uppercase. If the business wants case-insensitive input, fix the contract; if not, fix the code. Either way somebody must decide, and the decision is a requirement, which is why it belongs to the analyst rather than to whoever happens to be reading the CI log.

Every contract fix is also an input to contract testing: once the schema says enum: [EUR], consumers can rely on it. The habit of reading the contract this closely is the one I describe in the first time I read an OpenAPI contract, and you can rehearse the whole loop on a realistic payments contract in the free Analyze a Payment API lab.

The method for turning findings like these into a test suite and a defect backlog that a delivery team will act on is the core of API Testing and QA Mastery for BAs, and if the contract side is where your gaps are, API Fundamentals for Analysts covers reading schemas from the ground up.

How do you run Schemathesis in CI?

Use the official action, or install the CLI on a self-hosted runner inside the network. The action takes the schema, base URL, and authorization as inputs:

name: api-fuzz
on:
  pull_request:
    paths: ["openapi.yaml", "src/**"]

permissions:
  contents: read
  pull-requests: write

jobs:
  schemathesis:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: schemathesis/action@v3
        with:
          schema: openapi.yaml
          base-url: ${{ vars.FUZZ_BASE_URL }}
          authorization: "Bearer ${{ secrets.FUZZ_TOKEN }}"
          max-examples: "100"
          args: "--report junit --report-junit-path results/schemathesis.xml"
      - uses: actions/upload-artifact@v7
        if: ${{ !cancelled() }}
        with:
          name: schemathesis-results
          path: results/

Exit codes are simple: 0 all checks passed, 1 at least one check failed, 2 Schemathesis could not run (a broken schema or configuration). Pin the version (the action’s version input) so a new release does not change your gate.

Adopt it with a baseline. Point Schemathesis at an existing API and it will find twenty things on day one. If that blocks every pull request, the team will remove it by Friday. Record the known failures once, commit the file, and CI fails only on new ones:

st run openapi.yaml --url "$FUZZ_BASE_URL" --baseline schemathesis-baseline.json

# schemathesis.toml
baseline = "schemathesis-baseline.json"
headers = { Authorization = "Bearer ${FUZZ_TOKEN}" }
rate-limit = "auto"

[generation]
max-examples = 100

Then work the baseline down like any defect backlog. Where the suite sits next to your Bruno or Newman collection is the pipeline design in running API tests in CI.

What are the limits of API fuzzing?

• It creates data. Fuzzing POST /payments creates payments, possibly thousands. Run it only against a disposable environment with its own database, never against a shared SIT ledger, and never against a third-party sandbox you do not own. rate-limit = "auto" follows Retry-After on 429, but a rate limit is not permission.
• It only knows the contract. Schemathesis cannot know that a refund must not exceed the original payment amount unless you write that check. Business rules remain your hand-written cases, or custom checks in hooks.py.
• A bad contract gives bad tests. If the schema has no patterns, no maximums, and no enums, there is little to fuzz. Thin findings are often a contract-quality finding in themselves.
• Idempotency and concurrency are out of scope. It will not prove that two identical requests with the same Idempotency-Key create one payment; that needs the deliberate test in idempotency testing.
• Triage takes time. Budget for it, or the findings become noise.

Alternatives, briefly. Microsoft’s RESTler is an open-source stateful REST API fuzzer that infers producer-consumer dependencies from the OpenAPI definition, aimed more at security and reliability research. Dredd, the older OpenAPI conformance tester, has an archived GitHub repository with no release since 2021, so I would not start a new suite on it.

The takeaway

Schemathesis turns an OpenAPI contract into thousands of property-based tests: no 5xx, documented codes and schemas only, invalid input rejected, valid input accepted, auth enforced. Give it valid domain data such as IBANs through a hooks file, add custom checks for the business rules the contract cannot express, run it against a disposable environment, and adopt it in CI with a baseline. Then do the part no tool does: decide, finding by finding, whether the code or the contract is wrong, because that decision is a requirement.

Ahmed is a Senior Technical Business Analyst with 10+ years in banking and payments. He builds practical guides and tools for analysts at The Tech BA Toolkit.

Tags: API Testing, Fuzzing, OpenAPI, Property-Based Testing, QA