Skip to content

studio-design/openapi-contract-testing

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

355 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

openapi-contract-testing

OpenAPI Contract Testing for PHPUnit

CI Latest Stable Version Total Downloads PHP Version Require License

Framework-agnostic OpenAPI 3.0/3.1/3.2 contract testing for PHPUnit with endpoint coverage tracking.

Validate your API responses against your OpenAPI specification during testing, and get a coverage report showing which endpoints have been tested.

Features

  • OpenAPI 3.0, 3.1 & 3.2 support — Explicit version detection, including 3.2 QUERY, custom additionalOperations, form querystring, discriminator.defaultMapping, and observable streaming limitations
  • Response & request validation — dialect-aware JSON Schema via opis/json-schema: Draft 07 compatibility for OpenAPI 3.0 and native 2020-12 semantics for OpenAPI 3.1/3.2; application/json and any +json content type
  • Endpoint coverage tracking — Unique PHPUnit extension that reports which spec endpoints are covered by tests, at (method, path, status, content-type) granularity
  • Laravel route/spec parityopenapi:routes finds documented operations without routes and registered routes without OpenAPI operations, with filters, stable JSON, and independent CI gates
  • Schema-driven request fuzzing — Explore one endpoint or every supported operation with deterministic replay, filters, lifecycle/auth hooks, and explicit skip reasons
  • Enum drift detection — Static comparison between PHP backed enums and their enum: spec arrays, with PHPUnit-extension auto-discovery
  • Schema under-description detection — Optional strict mode that flags response fields the implementation always returns but the spec marks as optional, catching the spec gaps that conformance checks alone can't. See docs/strict-required.md for current scope and limitations.
  • Skip-by-status-code — Configurable regex list of status codes whose bodies are not validated (default: every 5xx); per-request via skipResponseCode()
  • PSR-7, Laravel, Symfony & Pest adapters — First-class PSR-7 request/response/exchange validation, auto-assert / auto-validate-request integration for Laravel, HttpFoundation assertions for Symfony, and Pest expectations
  • Parallel-runner safe — Coordinated sidecar+merge workflow for paratest / pest --parallel
  • Multi-format reports — Markdown / JUnit XML / JSON / HTML output with one-click GitHub Step Summary
  • Zero runtime overhead — Only used in test suites

Why this library?

Choose based on the workflow you need rather than on a single yes/no feature count:

  • Choose this library when you need response-level coverage at (method, path, status, content-type) granularity, several CI report formats, OpenAPI 3.1/3.2 JSON Schema semantics, schema-driven exploration, or drift detection across a framework-agnostic core and Laravel, Symfony, and Pest adapters.
  • Choose Spectator for a Laravel 12 application when generated test stubs, JSON assertion failures, or remote/private-GitHub spec sources matter more than response-level coverage granularity and broader framework support.
  • Choose league/openapi-psr7-validator when you want a low-level PSR-7 validator or PSR-15 middleware and will build the test/reporting integration yourself.
  • Choose osteel/openapi-httpfoundation-testing when you want a small HttpFoundation-to-PSR-7 validation bridge, or laravel-openapi-validator when automatic validation around Laravel HTTP tests is the main requirement.

Feature comparison (checked 2026-07-10)

Capability This library Spectator v3.0.2 league/psr7 v0.24 osteel v0.14 kirschbaum v2.0.2
OpenAPI versions explicitly supported 3.0, 3.1, 3.2 Version scope not stated 3.0.x 3+; delegates to League v0.22 Delegates to League v0.14–0.24
Request + response validation ✅ Laravel ✅ PSR-7 ✅ HttpFoundation / PSR-7 ✅ Laravel HTTP tests
Coverage granularity method, path, status, content-type method, path operation
Coverage outputs Markdown, JUnit XML, JSON, HTML, GitHub Step Summary Text, JSON
Parallel coverage merge Sidecar + merge CLI Not documented
Route/spec parity openapi:routes with text/JSON and CI gates spectator:routes
CLI diagnostics / scaffolding doctor, openapi:routes, coverage merge; no scaffolding validate, coverage, routes, stubs
Structured validation failures Text messages; JSON planned (#282) JSON {errors: [...]} PHP exception hierarchy Wrapper exception PHPUnit failure text
Schema-driven exploration Deterministic endpoint + whole-spec generation
Drift / under-description checks Enum drift, strict required
First-class integration PSR-7, Laravel, Symfony, Pest Laravel PSR-7, PSR-15 middleware HttpFoundation, PSR-7 Laravel auto-validation
Declared runtime floor PHP 8.2 core; Testbench 9–11 (Laravel 11–12; Laravel 13 / PHP 8.3) PHP 8.3, Laravel 12 PHP 7.2 PHP 8.0, HttpFoundation 5–8 PHP 8.0, Illuminate 10–13

Legend: ✅ supported · — no equivalent feature documented. “Not documented” is intentionally different from “unsupported”.

Methodology: This is a documentation/source audit, not a benchmark. Claims are limited to the linked, tag-pinned public documentation and Composer constraints checked on 2026-07-10. This-library claims describe main at 8c6416d; competitor versions are shown in the table header. Re-check this matrix using the release checklist at least quarterly or before a release when three months have elapsed.

Requirements

  • PHP 8.2+
  • PHPUnit 11, 12, or 13
  • A PSR-18 HTTP client + PSR-17 request factory (e.g. Guzzle, Symfony HttpClient) — only required when resolving HTTP(S) $refs

Installation

composer require --dev studio-design/openapi-contract-testing

YAML specs require symfony/yaml. It is listed under suggest so it isn't installed automatically. If your spec is JSON, you can skip this. If your spec is .yaml / .yml, add it explicitly:

composer require --dev symfony/yaml

Without it, the loader throws InvalidOpenApiSpecException with a clear "requires symfony/yaml" message the first time it tries to read a YAML file.

Quick start

Three steps to your first contract-tested endpoint. The example below uses a PSR-7 request and response; Laravel and Symfony integrations are documented in docs/setup.md.

1. Provide your OpenAPI spec

Point the loader at your spec's entry file. Internal and local-filesystem $ref are resolved automatically — no pre-bundling required:

openapi/
├── root.yaml          # paths reference ./schemas/*.yaml
└── schemas/
    ├── pet.yaml
    └── error.json

2. Register the PHPUnit extension

Before running your first test, verify that the package can load and enforce the contract:

vendor/bin/openapi-contract doctor \
  --spec=openapi/root.yaml \
  --strip-prefix=/api \
  --phpunit-snippet

The command resolves local references, checks the OpenAPI/JSON Schema dialect, reports unsupported enforcement features, counts discovered operations and responses, and exits non-zero for incompatible specs. Use --format=json in CI. See the doctor command reference for multiple specs, HTTP references, output categories, and exit codes.

Then register the emitted configuration:

<extensions>
    <bootstrap class="Studio\OpenApiContractTesting\PHPUnit\OpenApiCoverageExtension">
        <parameter name="spec_base_path" value="openapi/bundled"/>
        <parameter name="strip_prefixes" value="/api"/>
        <parameter name="specs" value="front,admin"/>
    </bootstrap>
</extensions>

3. Validate a PSR-7 exchange

When your application or HTTP client already returns PSR-7 messages, validate both sides and record coverage with one framework-independent call:

use Studio\OpenApiContractTesting\Psr7\OpenApiPsr7Validator;

$validator = new OpenApiPsr7Validator('front');
$result = $validator->validateExchange($request, $response);

$this->assertTrue($result->isValid(), $result->errorMessage());

The adapter accepts any psr/http-message implementation; no concrete PSR-7 package is added to production dependencies. A PHPUnit assertion trait, response-only operation addressing, PSR-15 test recipe, and stream guarantees are covered in the PSR-7 guide.

Laravel adapter

php artisan vendor:publish --tag=openapi-contract-testing

Set default_spec in the published config/openapi-contract-testing.php, then mix in the trait:

use Studio\OpenApiContractTesting\Laravel\ValidatesOpenApiSchema;

class GetPetsTest extends TestCase
{
    use ValidatesOpenApiSchema;

    public function test_list_pets(): void
    {
        $response = $this->get('/api/v1/pets');
        $response->assertOk();
        $this->assertResponseMatchesOpenApiSchema($response);
    }
}

Before running tests, compare Laravel's registered routes with the spec:

php artisan openapi:routes --fail-on-undocumented --fail-on-unimplemented

To validate every response automatically, set 'auto_assert' => true and drop the explicit assert call. To also catch request-side drift, set 'auto_validate_request' => true. See docs/setup.md for the full configuration and opt-out reference.

Documentation

Topic Reference
PSR-7 request / response / exchange validation and PSR-15 test recipe docs/psr7.md
Full setup, Laravel / Symfony / framework-agnostic adapters, auto-assert, opt-out attributes, request validation, HTTP $ref docs/setup.md
Pre-test compatibility diagnostics (openapi-contract doctor) docs/doctor.md
Laravel route/spec parity (openapi:routes) docs/laravel-route-parity.md
Pest plugin: expect()->toMatchOpenApiResponseSchema() and friends docs/pest-plugin.md
Schema-driven request fuzzing docs/fuzzing.md
Enum drift detection docs/enum-drift.md
Schema under-description detection (strict_required) docs/strict-required.md
Coverage report modes & threshold gate docs/coverage.md
HTML coverage output docs/coverage-html-output.md
JSON coverage output schema docs/coverage-json-schema.md
Parallel test runners (paratest / Pest --parallel) docs/parallel.md
CI integration (GitHub Actions, PR comments, output formats, partial-run handling) docs/ci.md
API reference (OpenApiResponseValidator, OpenApiSpecLoader, OpenApiCoverageTracker) docs/api-reference.md
Supported features, known limitations, warning channel docs/supported-features.md
Versioning policy & support matrix docs/versioning.md

Development

composer install

# Run tests
vendor/bin/phpunit

# Static analysis
vendor/bin/phpstan analyse

# Code style
vendor/bin/php-cs-fixer fix
vendor/bin/php-cs-fixer fix --dry-run --diff  # Check only

License

MIT License. See LICENSE for details.

About

Framework-agnostic OpenAPI 3.0/3.1 contract testing for PHPUnit with endpoint coverage tracking

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Packages

 
 
 

Contributors

Languages