Skip to content

[EX-2698] Document experiment protocols - #39064

Merged
drichards-87 merged 17 commits into
masterfrom
emiel.verkade/document-experiment-protocols
Aug 12, 2026
Merged

[EX-2698] Document experiment protocols#39064
drichards-87 merged 17 commits into
masterfrom
emiel.verkade/document-experiment-protocols

Conversation

@emielver

@emielver emielver commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

What does this PR do? What is the motivation?

Fixes EX-2698

Adds a page documenting experiment protocols, the reusable templates that set defaults for decision metrics, randomization, duration, notifications, and the statistical analysis plan.

The page covers:

  • Creating a protocol and configuring its six setup steps
  • Publishing a protocol
  • Creating an experiment from a protocol, and which defaults the protocol applies
  • Editing a published protocol
  • Duplicating, archiving, and deleting protocols

Also adds the page to the Experiments left navigation and a quick link on the Experiments overview page.

Merge readiness

  • Ready for merge

For Datadog employees:

  • ⚠️ Your branch name MUST follow the <name>/<description> convention and include the forward slash (/). If you've already created your PR with an incorrect branch name, please rename your branch and open a fresh PR.
  • 🤖 New: Comment with /review to run an automated check that catches common issues before a Documentation team member reviews your PR.

AI assistance

Drafted with Claude, then verified against the feature implementation and revised.

Additional notes

Opened as a draft while two screenshots are re-taken.

Add a page covering how to create, configure, publish, and apply
experiment protocols, plus duplicate, archive, and delete. Add the
page to the Experiments menu and the Experiments overview quick links.
@github-actions github-actions Bot added Architecture Everything related to the Doc backend Images Images are added/removed with this PR labels Aug 10, 2026
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Thanks for the thorough new page — it reads clearly and the alt text is excellent. A few minor style and clarity nits below; no blockers.

Reviewed 796d1b58a9869a85e979cbc263275aa9418b4bbfworkflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Clarity nit on the Randomization step's Environment field. The rest of the page reads well and follows the Datadog style guide.

Reviewed 4a6747c0b806af2b9030b24a5afe51405da37180workflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Nice, thorough walk-through of protocols. A few small suggestions inline — mostly copy polish and one capitalization to double-check against the UI.

Reviewed e91945e5963939a5435669d8fdd77666e7138689workflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Emiel Verkade <emiel.verkade@datadoghq.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Overall well-structured page with clear steps and good use of alt text. A few small findings on link style and clarity of a couple of sentences. No blockers.

Reviewed 604d2d90093a4e24a0dc855f7a7a7729869c62ecworkflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/_index.md
@jhgilbert jhgilbert added the astro-reorg-no-conflicts Needs manual conflict resolution after replatforming reorg label Aug 10, 2026
emielver and others added 2 commits August 11, 2026 11:16
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Emiel Verkade <emiel.verkade@datadoghq.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Well-written new page. Structure and menu integration look good. Just a couple of minor nits — no blockers.

Reviewed c19f7406d3eedf928568c225c6d6f0d2b5c49e0dworkflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Emiel Verkade <emiel.verkade@datadoghq.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

New page and menu entry for Experiment Protocols. Content is well-structured with clear steps, prerequisites, and role/permissions notes. One style blocker (raw HTML) and a couple of minor phrasing nits below.

Reviewed b4eeea3db127672b434a99ffcb9fa50567b31794workflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Emiel Verkade <emiel.verkade@datadoghq.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Solid, thorough new page. A couple of small clarity nits around fields marked (Optional) that are effectively required for a downstream step in the same numbered list — noted inline.

Reviewed f822e3908f86a1a6ce893067087098d26393af52workflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md
Comment thread hugo/content/en/experiments/protocols.md
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Co-authored-by: Emiel Verkade <emiel.verkade@datadoghq.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Thorough new page for a substantial feature. Content is well-structured with clear step-by-step guidance, good use of the {{< ui >}} shortcode for UI labels, and thorough alt text on images. The menu weights are correctly renumbered to accommodate the new page and to resolve a pre-existing duplicate weight. A few minor style and clarity nits — nothing blocking.

Reviewed b7296cb3853d012d6c42886ab3a8dd814a942c86workflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Emiel Verkade <emiel.verkade@datadoghq.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Overall a well-structured page. Two things worth fixing: a duplicated image shortcode in Step 4, and a use of "will" that trips the style guide's temporal-words rule. A few minor nits inline.

Reviewed badc70287f740c13e74af9ceaa81d2b88bd2b6b9workflow run

Comment thread hugo/content/en/experiments/protocols.md
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md Outdated
Comment thread hugo/content/en/experiments/protocols.md
Comment thread hugo/content/en/experiments/protocols.md
Comment thread hugo/content/en/experiments/protocols.md
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Emiel Verkade <emiel.verkade@datadoghq.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Overall this is a clear, well-structured new page. One nit: the [11] link reference to the permissions page is defined at the bottom of the file but never used inline in the Prerequisites section.

Reviewed b0a72a1f22276e4fb9c0934133e7f76504af190eworkflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
emielver and others added 2 commits August 11, 2026 13:50
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@emielver

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review by Claude. AI-generated; verify before acting.

Well-written new page. Content is clear, structured logically, and follows the style guide (no forbidden words detected). Menu entry, frontmatter, description, and reference-style links all look correct. Weight renumbering in main.en.yaml also cleans up a pre-existing duplicate (pa_experiments_statistics and pa_experiments_metric_property_breakouts were both weight 6). One minor style nit inline.

Reviewed 0c0561032c87decb56b847d3bde1fe8f38f52fcdworkflow run

Comment thread hugo/content/en/experiments/protocols.md Outdated
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@emielver
emielver marked this pull request as ready for review August 11, 2026 13:00
@emielver
emielver requested a review from a team as a code owner August 11, 2026 13:00
@drichards-87 drichards-87 added the editorial review Waiting on a more in-depth review label Aug 11, 2026
@drichards-87

Copy link
Copy Markdown
Contributor

Created DOCS-15373 for Docs Team review.

@drichards-87 drichards-87 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emielver Looks good overall. I just left one piece of feedback on a change we need to make before we merge the PR.

@emielver
emielver requested a review from drichards-87 August 12, 2026 04:38
@drichards-87
drichards-87 merged commit 782a0b7 into master Aug 12, 2026
21 checks passed
@drichards-87
drichards-87 deleted the emiel.verkade/document-experiment-protocols branch August 12, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Architecture Everything related to the Doc backend astro-reorg-no-conflicts Needs manual conflict resolution after replatforming reorg editorial review Waiting on a more in-depth review Images Images are added/removed with this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants