feat: add DQDL support via EvaluateDataQuality (#205)

[AMC-hawk]

Issue #, if available: Closes #205

Description of changes:

Deequ already supports data freshness as a DQDL rule (awslabs/deequ#661, in every release since 2.0.14, including the 2.0.21 jar PyDeequ pins). PyDeequ doesn't expose DQDL, so Python users can't use it. This PR adds a thin wrapper around com.amazon.deequ.dqdl.EvaluateDataQuality.

from pydeequ.dqdl import EvaluateDataQuality

ruleset = 'Rules=[IsComplete "id", DataFreshness "updated_at" <= 24 hours]'

EvaluateDataQuality.process(spark, df, ruleset)      # one row per rule
EvaluateDataQuality.processRows(spark, df, ruleset)  # originalData / ruleOutcomes / rowLevelOutcomes

Changes

• pydeequ/dqdl.py: EvaluateDataQuality.process and processRows, matching the Scala API. Supports additionalDataSources={"alias": df} for rules like RowCountMatch, pandas=True output, and pandas input (same as VerificationSuite.onData).
• tests/test_dqdl.py: output schema, passing and failing DataFreshness rules (including the compliance metric), days/minutes units, additionalDataSources, pandas output, row-level outcomes, and input validation.
• README.md and docs/source/pydeequ.rst: usage section and API docs.

No changes to existing APIs or the Deequ dependency.

Notes

• DataFreshness doesn't have row-level support in Deequ, so processRows lists it under DataQualityRulesSkip. The README says this.
• Spark 3.5 logs ANTLR version mismatch warnings (4.7 vs 4.9.3) when the DQDL parser loads. Parsing works; the warnings come from the upstream dqdl dependency.

Testing

pytest tests/test_dqdl.py tests/test_verification.py: 15 passed locally (Spark 3.5.9, Python 3.11, Java 17, Deequ 2.0.21-spark-3.5). black and flake8 pass on the new files.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

---

Generated by AI (model: us.anthropic.claude-opus-4-8, prompt: 295226dd) — may not be fully accurate. Reply if this doesn't help.
Reviewed by Shadow · github.com/sudsali/shadow

[AMC-hawk]

@sudsali Can you please review it. Thanks

[github-actions]

No issues found.

---

Generated by AI (model: us.anthropic.claude-opus-4-8, prompt: 295226dd) — may not be fully accurate. Reply if this doesn't help.
Reviewed by Shadow · github.com/sudsali/shadow