Skip to content

feat(evaluate): Add Azure AI Evaluation & Observability Workshop Notebook - #200

Open
Govind Kamtamneni (thegovind) wants to merge 9 commits into
Azure-Samples:mainfrom
thegovind:main
Open

feat(evaluate): Add Azure AI Evaluation & Observability Workshop Notebook#200
Govind Kamtamneni (thegovind) wants to merge 9 commits into
Azure-Samples:mainfrom
thegovind:main

Conversation

@thegovind

Copy link
Copy Markdown

Description

This PR adds a comprehensive workshop notebook demonstrating Azure AI's evaluation and observability capabilities, developed in collaboration with Chang Liu (@changliu2) and Alex Sutton for TechConnect. This is the first iteration of the workshop material, which has been fully tested and provides hands-on examples of:

  • Safety evaluation using ContentSafety, Violence, and HateUnfairness evaluators
  • Security testing with DirectAttackSimulator and adversarial testing
  • Quality assessment using Relevance, Coherence, and Groundedness evaluators
  • Cloud-based evaluation workflows with azure-ai-projects
  • Observability setup with OpenTelemetry and Application Insights
  • Best practices for responsible AI governance

The workshop includes:

  • A Jupyter notebook with detailed explanations and working code
  • Sample evaluation test data in JSONL format
  • Environment configuration templates

This serves as a practical guide for developers to implement comprehensive evaluation and monitoring in their Azure AI applications.

Checklist

  • I have read the contribution guidelines
  • I have coordinated with the docs team (mldocs@microsoft.com) if this PR deletes files or changes any file names or file extensions.
  • This notebook or file is added to the CODEOWNERS file, pointing to the author or the author's team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants