Skip to content

Add experimental certificate search to the KMS layer - #1089

Open
dopey wants to merge 10 commits into
masterfrom
feat/add-certificate-searcher
Open

Add experimental certificate search to the KMS layer#1089
dopey wants to merge 10 commits into
masterfrom
feat/add-certificate-searcher

Conversation

@dopey

@dopey dopey commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Adds an experimental CertificateSearcher KMS interface and its CAPI (Windows) implementation: enumerate a certificate store and return each certificate together with its CNG key-container name, read from the CERT_KEY_PROV_INFO property rather than by opening the key — so certificates whose private key no longer exists are still reported and can be identified for cleanup.

Motivation: a Windows certificate store can accumulate certificates whose backing key is gone. Such a certificate still reports HasPrivateKey=True (the property survives key deletion), still looks healthy in certlm.msc, and can still be selected by schannel for EAP-TLS — failing only at handshake time. Identifying these requires reading the key-container association from certificate properties without opening the key, which no current KMS API exposes.

Alternative considered: extending CleanupCredentials with a non-expired mode. Rejected because deciding which unexpired certificates to remove is caller policy — expressing it would mean the API absorbing caller naming schemes or accepting pattern arguments — and because CleanupCredentials is safe by construction today (expired-only means it cannot destroy a working credential), a property worth keeping. Search + the existing DeleteCertificate keeps the read side harmless and the destructive operation precise. A composite cleanup could still be layered on this primitive later.

What's in here

  • kms/apiv1: CertificateSearcher (embeds KeyManager, matching the SearchableKeyManager precedent rather than the bare-interface CredentialsCleaner style) + request/response types, all marked Experimental
  • kms/capi: SearchCertificates implementation; a fix for cryptFindCertificateKeyContainerName, which previously passed a nil buffer on its second property call and unconditionally returned an empty name (it had no callers, so nothing depended on the broken result)
  • kms/capi: certContextToX509 now clones the DER before parsing. x509.ParseCertificate aliases its input, so the helper previously handed out certificates whose Raw/RawSubject/etc. pointed into store-owned memory — freed on the next enumeration advance, or kept alive only by unclosed store handles. This is a strict correctness improvement for the pre-existing callers (LoadCertificate, FindCertificatesByIssuer, CleanupCredentials) as well.
  • kms/capi: runtime.KeepAlive in the container-name reader — the name is an interior pointer into a []byte the GC scans as pointer-free, so the buffer must be pinned across the UTF-16 walk
  • kms/platform: pass-through following the CleanupCredentials pattern, with per-OS tests

Design decisions worth calling out

  • SearchCertificates closes its store handle; the file's five other CertOpenStore sites do not (pre-existing, deliberately untouched). A search API invites frequent calls from long-running processes, where an unclosed handle per call accumulates without bound; changing the other sites' behavior was out of scope here.
  • Partial results on error: a mid-enumeration failure returns the certificates gathered so far alongside the error, documented on the interface and the wrapper (same contract discussion SearchKeys went through).
  • Per-certificate failures skip (unparseable DER, unreadable properties) rather than failing the search — best-effort metadata by contract.

Verification

CI here runs ubuntu-only, so none of the windows-tagged code executes in CI — stating exactly what was run locally:

  • go build ./..., go test ./kms/... (host; the new per-OS NotImplementedError tests run here)
  • GOOS=windows GOARCH=amd64 go build ./kms/... and go vet — clean; the one vet finding (ncrypt_windows.go possible misuse of unsafe.Pointer) is pre-existing at the merge base in an untouched function
  • GOOS=windows GOARCH=amd64 golangci-lint run ./kms/capi/... ./kms/platform/... — zero new findings vs the merge base
  • Windows-tagged tests (kms/capi/capi_windows_test.go, the new delegation test in kms/platform/kms_windows_test.go) compile under GOOS=windows and run on Windows machines; the delegation test asserts the searched certificate's recorded container name against the stored value, so it distinguishes the fixed property-reader from the stub it replaced

🤖 Generated with Claude Code

dopey and others added 6 commits July 27, 2026 13:28
An optional interface for KMS implementations that can enumerate a
certificate store's contents together with each certificate's private-key
association. The key-container name is reported from certificate metadata
rather than by opening the key, so callers can identify which certificates
belong to which keys — including certificates whose private key no longer
exists.

Change-Type: feature
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The second CertGetCertificateContextProperty call passed a nil,
never-allocated buffer, and the function unconditionally returned an
empty name. Nothing called it, so no existing behavior depended on the
broken result.

Replace the body with the two-call size-then-data protocol the sibling
property helpers use, distinguishing a certificate with no key
association (CRYPT_E_NOT_FOUND, empty name) from genuine read failures.

Change-Type: fix
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Enumerates the certificate store selected by the request URI's
store-location and store attributes and returns each certificate
together with its key-container name, read from the CERT_KEY_PROV_INFO
property rather than by opening the key — so certificates whose key no
longer exists are still reported. A mid-enumeration failure returns the
certificates gathered so far alongside the error, and a certificate
whose DER cannot be parsed or whose properties cannot be read is
skipped rather than failing the search.

Change-Type: feature
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…re handle

x509.ParseCertificate aliases its input, so certContextToX509 handed out
certificates whose Raw, RawSubject, and related fields pointed into
store-owned memory — freed by the next enumeration advance, or kept
alive only by store handles that were never closed. Clone the DER before
parsing; every caller of the helper gets certificates backed by
Go-owned memory.

SearchCertificates now closes its store handle. It runs on every
certificate issuance in long-running callers, where an unclosed handle
per call accumulates without bound.

Change-Type: fix
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Delegates to backends implementing CertificateSearcher, applying the
same request-URI translation CleanupCredentials uses so callers can
address the platform scheme uniformly. Platforms whose backend lacks
the interface report NotImplementedError.

Change-Type: feature
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keep the property buffer alive across the UTF-16 walk in
cryptFindCertificateKeyContainerName: the container name is an interior
pointer into a buffer the garbage collector treats as pointer-free, so
without runtime.KeepAlive the buffer could be collected mid-read.

Document at the interface and the platform wrapper that a failed search
may still return the certificates enumerated before the failure, and
assert the recorded container name in the delegation test so the
property read is verified against a known value. Also: singular
SearchCertificateResult to match the SearchKeys family, experimental
notices on the payload types, unsafe.Slice in place of the deprecated
reflect.SliceHeader, and test cleanup registered via t.Cleanup
immediately after the store write.

Change-Type: fix
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dopey
dopey marked this pull request as ready for review July 27, 2026 22:58
dopey and others added 2 commits July 27, 2026 18:41
The platform KMS selects the TPMKMS backend for default \"kms:\" URIs on
Windows (kms_windows.go's newKMS routes PlatformKMS, DefaultKMS, and TPMKMS
alike to newTPMKMS), not CAPIKMS. TPMKMS had no SearchCertificates method, so
the platform wrapper's apiv1.CertificateSearcher type assertion failed and
returned apiv1.NotImplementedError{} for the exact configuration production
consumers use. On real hardware this made a certificate sweep built on
SearchCertificates a silent no-op: superseded certificates were never found,
so they were never deleted.

Add TPMKMS.SearchCertificates, mirroring CleanupCredentials: it requires the
Windows certificate store to be configured, reads store-location/store
overrides from the request URI (falling back to the instance's configured
store), and delegates to the same windowsCertificateManager (capi.CAPIKMS on
Windows) used by CleanupCredentials, LoadCertificateChain, and
StoreCertificateChain. The response is returned unmodified so the
partial-results contract (a non-nil response alongside a non-nil error on a
mid-enumeration failure) flows through untouched.

Extend the capiCertificateManager interface with SearchCertificates; the only
test fake implementing it (fakeWindowsCertificateManager, gated behind the
tpmsimulator build tag) gets a no-op implementation to keep compiling.

Tests: a host-runnable tpmkms test asserts a TPMKMS not configured for the
Windows certificate store (the only possible state off Windows) returns
NotImplementedError. A new Windows test in kms/platform constructs the
platform KMS the same way the production consumer does - a default \"kms:\"
URI with no backend argument - confirms it resolves to TPMKMS, and exercises
store/search/delete end to end; it requires a TPM with the Windows Platform
Crypto Provider and skips via the file's existing SkipTests() guard when none
is available.

Change-Type: fix
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review follow-up on the SearchCertificates fix: the only test reaching
TPMKMS.SearchCertificates's delegation body was Windows+TPM-gated, so it runs
on zero CI runners. The downstream capi URI construction (store-location and
store attribute spellings) is exactly where a typo would silently search the
wrong certificate store, reproducing the original symptom, and nothing
caught that on ubuntu CI.

Add TestTPMKMS_SearchCertificates_delegation (tpmkms_search_test.go), a
host-runnable test using a recording capiCertificateManager fake, no TPM or
Windows required. It asserts the request Name TPMKMS hands its Windows
certificate manager parses to store-location=machine;store=My when the
caller's URI carries those overrides (mirroring what the platform wrapper's
transformToTPMKMS produces for "kms:store-location=machine;store=My"), and
that it falls back to the instance's configured defaults when the caller's
URI carries none. This pins hop 8 of the delegation on every CI run.

Fix fakeWindowsCertificateManager.SearchCertificates (tpmkms_simulator_test.go,
gated behind the tpmsimulator build tag): it previously returned (nil, nil),
the exact shape the original bug produced, so a future test written against
it could assert nothing and still pass. It now records the request it
received and returns a canned non-nil response with one result, unless a
test configures searchResp/searchErr explicitly.

TestKMS_SearchCertificates_platform (kms_windows_test.go) is left using the
user/My defaults rather than store-location=machine;store=My: writing to the
machine store needs administrator privileges, which the TPM-equipped test
runner isn't guaranteed to have, and store/search/delete need to agree with
each other. The override branch (tpmkms.go's SearchCertificates,
location/store fallback) is covered instead by the new host-runnable
delegation test above, which runs unconditionally on ubuntu CI rather than
only on a TPM-equipped Windows box.

Also note in TPMKMS.SearchCertificates's doc comment that the platform
wrapper's URI transform unconditionally injects
skip-find-certificate-key=true into every Windows request, including
searches; it's parsed but unused on this path since it only matters for
certificate-to-key association during store.

Change-Type: fix
Release-Note: no
Audience: developer
Impact: low
Breaking: false
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dopey
dopey requested review from darkfronza, joshdrake and maraino and removed request for maraino July 28, 2026 04:19
@dopey

dopey commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Validation beyond what CI can exercise: the full production path — platform KMS constructed from a default kms: URI (TPMKMS backend) → capi → LocalMachine\My — was run end-to-end on TPM-equipped Windows VMs. Both target scenarios pass: superseded-certificate cleanup on re-issuance, and the orphaned case — a deployed certificate whose CNG key no longer exists, found by container name via the property read and deleted by thumbprint. The TPMKMS.SearchCertificates commit exists because that end-to-end run caught the default-URI routing going to the TPM backend, which the original capi-only implementation never received; the host-runnable delegation tests added alongside pin that seam in ubuntu CI permanently.

@dopey
dopey requested review from hslatman and maraino July 28, 2026 04:36

@maraino maraino left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good, but I think is too CAPI-specific, I've suggested a change that might make it more generic.

Comment thread kms/apiv1/requests.go
// release.
type SearchCertificateResult struct {
Certificate *x509.Certificate
KeyContainerName string

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is the key name, right? At least in capi, the keyContainerName in the URI comes from the key parameter.

If this is the case, why don't we make this more generic and rename it to KeyName?

@darkfronza darkfronza left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two minor comments

Comment thread kms/capi/capi.go

x509Cert, err := certContextToX509(certHandle)
if err != nil {
continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should errors.Join this error with any relevant information from certHandle, silently ignoring it will make debugging harder.

Comment thread kms/capi/capi.go

containerName, err := cryptFindCertificateKeyContainerName(certHandle)
if err != nil {
continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should errors.Join this error with any relevant information from certHandle, silently ignoring it will make debugging harder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants