Conversation
Implement format=bundle on the snapshot retrieval endpoints (GET /api/release-tracks/:id and .../snapshots/:modified): - include (staged/candidates) hydrates additional tiers alongside members; state (work-in-progress/awaiting-review) narrows them, with reviewed entries always included - stixVersion (2.0|2.1, default 2.1) conforms objects via the shared lib/stix-conformance.js helpers (extracted from stix-bundles-service); the bundle envelope carries spec_version only for STIX 2.0 per spec - includeToc (default true) prepends an x-mitre-collection table of contents derived from the release-track metadata - bundles are self-contained (referenced identities and marking definitions included), LinkById tags are converted to citations, and notes are never emitted Rework the ephemeral endpoint (GET /api/release-tracks/ephemeral/:domain) to delegate bundle generation to stix-bundles-service, preserving the legacy object-selection logic (secondary objects, referential integrity, STIX conformance) with a simplified parameter surface: includeToc, includeObjectsWithMissingAttackId, includeDeprecated, includeRevoked, stixVersion. The legacy GET /api/stix-bundles endpoint is marked deprecated in the OpenAPI spec. Add regression tests (running with ADM validation enabled), update the OpenAPI spec, and document the behavior in the user docs and in docs/developer/release-tracks/bundle-export.md.
The identity, namespace, and static bypass-rule seeding paths inserted rules and relied on the unique (fieldPath, errorCode, stixType) index to reject duplicates via DuplicateIdError. On a fresh database the index builds in the background, so a duplicate insert could succeed before the index existed; a later explicit index build (e.g. Model.init()) then failed with E11000. Seed rules with an upsert keyed on the compound index fields instead, which is idempotent regardless of index state.
Add AGENTS.md as the canonical, machine-independent agent guide: workspace conventions, architecture map, validation layers, task workflow (docs-first, TODO.md scratchpad, strict test verification, definition of done), regression- test recipes, Bruno conventions, known gotchas, and guide-maintenance rules. Machine-specific paths (workspace parent, ADM source checkout, Bruno collection) live in a gitignored AGENTS.local.md; commit AGENTS.local.example.md as the copyable template. CLAUDE.md imports both so Claude Code and other coding agents share a single source of truth.
Fix two flake sources in the mocha suites: - Reuse a single mongodb-memory-server instance for all spec files in the process. Per-file stop/start intermittently failed with "Port already in use" (a port not yet released by the previous instance), breaking that file's before() hook and cascading failures through the whole file. closeConnection now drops the database and disconnects but keeps the server running; the mocha scripts run with --exit. - Explicitly rebuild schema indexes after each reconnect. Dropping the database also drops its indexes, and mongoose's per-model init() is memoized per process, so unique-index constraints (stix.id + stix.modified) intermittently vanished for later spec files, letting duplicate-POST tests and dependent count assertions fail in roaming pairs. Also records the (now historic) flake signature in the AGENTS.md gotchas.
Add workspace.release_tracks backrefs ([{ id, tier, status }]) to STIX
object documents so release-track membership is visible from any standard
object getter without scanning tracks. Tier values match the snapshot tier
array names (members/staged/candidates/quarantine), consistent with the
tier field on the object-versions endpoint.
Backrefs are maintained by snapshot-driven reconciliation: every snapshot
persistence choke point (cloneSnapshot, track clone, snapshot/track
deletion, bump) emits release-track::contents-changed with the track's
latest snapshot; AttackObjectsService and RelationshipsService listeners
diff desired vs current entries and bulk-write the difference. The design
is idempotent and self-healing, covering all membership mutation routes
with one code path. Sparse multikey indexes on workspace.release_tracks.id
support the reverse lookups.
The field is server-controlled: stripped from client create/update/import
input and excluded from every revision-clone path (revoke, relationship
deprecation, technique conversion, identity propagation).
The regression suite for this feature lands at the end of this commit
series (release-tracks-backrefs.spec.js), since it also covers the
follow-up behavior changes that build on the same infrastructure.
PUT merged body stix.id/stix.modified over the stored document, so an update could silently re-key a revision — stranding release-track pins and orphaning workspace.release_tracks backrefs. updateFull now returns 400 when the body identity fields differ from the path parameters; re-keying must go through POST, which creates a new revision that release-track revision sync captures. Rewrites the legacy PUT regression tests across all SDO types, which encoded the old bump-modified-through-the-body convention. The Angular frontend is unaffected: its PUT factory always serializes the body with the same modified value used in the URL.
…isions Member sync only watched the members tier, so creating a new revision of a candidate- or staged-pinned object silently stranded the pin on the old revision: the release would ship stale content and the object's latest view lost its workspace.release_tracks backref. Under track_latest, pins in all three tiers now follow new revisions per the supplant config (replace moves the pin, queue adds a second candidate, ignore skips); manual tracks are unchanged. This reverses the documented members-only scope — see the behavior evolution note in member-sync-strategies.md. Also make create/update responses read their own writes: the awaited created/updated events can re-pin a track to the new revision, so BaseService now refreshes workspace.release_tracks after event processing instead of returning a response composed before the backref was stamped.
Manual candidate adds and demotions appended blindly when the object_ref was already pinned in candidates at a different revision, duplicating candidates. Entries into the candidates tier now pass through the same conflict machinery as the other tier transitions, governed by config.promotion_conflicts.into_candidates (prefer_latest by default; abort returns 409). Exact (object_ref, object_modified) re-adds remain idempotent, and revision-sync enrollment keeps its own supplant semantics. Also adds the regression suite covering the whole release-track backref series: lifecycle backrefs across every tier transition, server-controlled stripping (create/update/import and all revision-clone paths), revision sync for member/candidate/staged pins, read-your-own-writes responses, manual re-adds, and the conflict policies.
Legacy deleteById/deleteVersionById handlers in 13 controllers caught all service errors and returned a blanket 500, hiding typed exceptions from the centralized error handler. Migrate them to next(err) per service-exception-middleware.md; also removes stray console.log debugging in the software controller.
Members-pinned revisions are released content: PUT and DELETE (single version or all versions) now return 409 (MemberPinnedRevisionError) when any release track pins the revision in its members tier, with guidance to create a new revision instead (x_mitre_deprecated to retire). Adds the 409 responses to the affected OpenAPI operations.
…on sync Two blind spots closed: (1) member sync now subscribes to the per-type ::revoked events, so the revoked revision is enrolled as a candidate in member tracks and candidate/staged pins move to it — previously a track silently kept exporting the pre-revoke revision; the revoke response carries the resulting backrefs. (2) In-place PUTs of candidate/staged- pinned revisions reset the entry for re-review (staged demotes to candidates), including in-place deprecation; enrollment of already-pinned revisions and no-op snapshot clones are suppressed, fixing the same-key duplicate cross-tier enrollment misfire.
Centralize tier/status placement in a workflow gate (lib/release-tracks/workflow-gate.js): given the trigger (new revision, in-place edit, revocation), the entry's previous state, and the track config, the gate decides where a synced entry lands. The candidacy threshold is codified into placement — qualifying entries go directly to staged in a single snapshot instead of bouncing through candidates via a post-hoc auto-promotion pass, and workflow-service shares the gate's status ranking. In-place PUTs of pinned revisions now mark the entry with the server-assigned modified-in-place status: the content changed but carries no revision history, so reviewers are told that a re-review is required without pretending to know what changed. The marker ranks with work-in-progress, so permissive tracks (candidacy_threshold work-in-progress) keep in-place-edited staged entries staged, while strict tracks demote them to candidates. Cleared via the review endpoint (from: modified-in-place); never carried onto new revisions by the preserve policy.
Technique/subtechnique conversions save the new revision directly via the repository, so no created/updated event fired and a track pinning the converted object kept pinning the pre-conversion revision. The conversion events now carry the converted revision and acting user, and member sync subscribes via an adapter (same pattern as the revoked events), treating the conversion as a new-revision trigger through the workflow gate: candidate/staged pins move to the converted revision and member tracks enroll it as a candidate. Conversion responses refresh workspace.release_tracks after event processing so the result carries the re-pinned backrefs.
…refs Backref entries now carry the referencing track's type (standard or virtual), so consumers can distinguish a virtual release's reference from a standard track's without fetching the track. Entries written before the field existed are backfilled on the track's next contents change. Adds the first regression coverage of virtual-track backrefs (composition over a tagged component track).
Maintain tagged snapshot references in the release-track registry and query historical membership across per-track collections with bounded concurrency. Backfill existing tracks, protect tagged snapshots, and document and test the new endpoint.
Normalize exact STIX revision pins across release-track tiers and make exact transitions idempotent. Repair legacy duplicate state during mutations and tagging, with regression and documentation coverage.
Add paginated snapshot history and canonical latest-snapshot endpoints. Remove implicit latest retrieval from the release-track resource path and migrate API consumers, tests, documentation, and Bruno requests.
Replace bump routes with snapshot-scoped release endpoints and shared summary, workbench, and bundle planning. Resolve latest snapshots at request time and use modified paths to target specific snapshots
Scope virtual composition and draft creation beneath the virtual namespace, remove the standalone composition preview, and share persisted snapshot release previews across track types. Add chronological virtual release summaries, functional domain filters, relationship-complete bundle exports, regression coverage, OpenAPI updates, and lifecycle documentation.
Invalidate materialized contents when composition changes and reject release planning until the virtual draft is rematerialized. Restrict direct contents replacement to standard tracks and document the remaining virtual-track work.
Add an explicitly virtual-scoped endpoint for selecting an exact quarantined revision into a new draft. Preserve materialization provenance, reconcile back-references, and update the API contract and regression coverage.
Reject unknown composition properties and enforce strategy-specific component selectors across virtual-track creation and updates. Align OpenAPI, documentation, frontend guidance, and Bruno examples.
Require unique component priorities and track IDs, validate referenced standard tracks before initial virtual-track persistence, and align request, persistence, OpenAPI, documentation, and frontend contracts.
Enforce strict mode-specific virtual snapshot schedules across request, service, persistence, OpenAPI, documentation, and frontend contracts. Reject schedule metadata for standard tracks and record required cron and explicit-date scheduler work.
Constrain virtual component object-type filters to the canonical Workbench STIX vocabulary across request, service, persistence, OpenAPI, and documentation boundaries. Preserve exact component snapshot revisions.
Collapse exact component revision duplicates before resolving conflicts, attribute every surviving member to one deterministic source, and quarantine only genuinely different revisions.
Persist immutable component track versions from the materialized virtual draft in release history, validate the provenance map, and align API, documentation, frontend, and regression contracts.
Execute persisted cron and date schedules through the existing virtual snapshot lifecycle. Add durable occurrence claims, restart-safe idempotency, automation-run auditing, retry behavior, scheduled snapshot provenance, and aligned API and operations documentation.
Preserve latest selectors through candidate and staged workflow state, resolve them during standard release planning, and persist exact immutable members. Align virtual materialization, enrichment, exports, backrefs, schemas, tests, clients, and documentation with the deterministic membership boundary.
Repair the legacy collection-index scheduler regression, expand cron and date schedule coverage, recover persisted snapshots after interrupted occurrences, and include scheduler tests in the default test gate.
Enforce tagged-version uniqueness with a database partial index, return a typed conflict for concurrent release races, and add a fail-closed migration for existing and orphan track collections.
Validate exact primary content at request, release, clone, materialization, import, and export boundaries. Return structured missing references instead of persisting or rendering partial release data.
Clear per-track Mongoose models when the in-memory database is dropped so later specs do not recreate indexes for collections that no longer exist.
Guard tagged revisions from authoritative snapshot history and make backref reconciliation required, durable, observable, and repairable after partial failures.
Require administrator authorization and exact track confirmation for member replacement and full-track deletion. Persist actor-attributed audit events before execution and expose durable identifiers when finalization fails.
Retain database-enforced release version uniqueness while treating existing beta track collections as disposable development state.
Persist exact primary, relationship, and secondary revisions in snapshot graph manifests. Pin relationship endpoints, protect referenced revisions, and add migration tooling for existing data. Make persisted snapshot history immutable by removing direct contents and historical metadata mutation routes. Limit snapshot deletion to the latest untagged draft and align the API contract, tests, and documentation.
…on-readiness Fix/release tracks production readiness
Ignore deprecated and revoked dangling relationship history during deterministic graph backfill while preserving strict validation for active relationships. Improve migration diagnostics and ensure manifest indexes are always established.
…graph-migration fix(release-tracks): scope graph migration to active relationships
Permit ampersands in release-track names across request, persistence, and OpenAPI validation. Add regression coverage and document the accepted naming contract.
Validate caller-supplied configuration using the existing track config contract and persist it on the initial release-track snapshot.
Infer domain unions from persisted canonical collection provenance and cover every domain-bearing ATT&CK type. Let the native migration driver generate inactive clone IDs to prevent cross-version BSON failures.
Accept scheduled materialization metadata on virtual track creation, composition updates, and explicit materialization. Expose persisted values through track and snapshot retrieval endpoints.
Add virtual-specific API coverage confirming the existing shared bundle serializer emits STIX 2.0 on request and preserves the STIX 2.1 default. Clarify the established behavior in OpenAPI and release-track documentation.
Reuse the captured graph manifest across virtual release previews and commits so tagged bundles retain the audited materialization. Apply component domain filters to secondary traversal and ignore inactive LinkById collisions.
enable stixExport and readOnly service role access to /snapshots/latest endpoint
Add tagged-snapshot graph create and delete endpoints with pointer-only member manifests and live graph fallback. Make persisted STIX revisions immutable, retain one rolling standard draft, and expose graph statistics in snapshot history.
Reconstruct v19.1 manifests from exact persisted entity and relationship revisions, including deterministic LinkById dependencies and source serialization hints. Fail closed when canonical-domain provenance is unavailable and provide a corrective migration for previously inferred revisions.
Convert Mongoose Date values to ISO timestamps before URL encoding so the virtual graph regression addresses the intended conflict path.
Persist bounded snapshot-local descriptions across creation, materialization, and release workflows. Add an editor-authorized update endpoint with OpenAPI documentation and regression coverage.
Select relationships through exact revision-pinned endpoints and require both endpoints to be snapshot members. Carry forward source-attested predecessor relationships while preventing secondary revision leakage and duplicate STIX objects.
Map snapshot descriptions onto exported collection TOCs. Enforce chronological release bounds and serialize release commits.
Freeze the collection object in deterministic graph manifests and store SHA-256 values for exact STIX 2.0 and 2.1 downloads. Reject snapshot-note edits while a bundle cache exists so the reported hashes remain valid.
Use a track-stable collection ID and the configured organization identity for STIX 2.1 table-of-contents objects. Omit collection objects from STIX 2.0 exports and migrate existing graph entries and bundle hashes.
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## next #492 +/- ##
==========================================
+ Coverage 71.10% 80.15% +9.05%
==========================================
Files 220 242 +22
Lines 31305 38153 +6848
Branches 2941 4646 +1705
==========================================
+ Hits 22260 30583 +8323
+ Misses 9003 7483 -1520
- Partials 42 87 +45 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.