feat(#11376): add docs for automatic archiving

content/en/technical-overview/data/db-schema.md

@@ -253,6 +253,7 @@ Users then, can be represented by up to 3 docs:
 * updated only after the data for their emitting contact changes or every 7 days
 * created in the database for any task due within the last 60 days
 * immutable once their state is "terminal" (Cancelled, Completed, Failed)
+* [archived automatically](/technical-overview/data/performance/archiving/#automatic-archiving) once terminal and more than 60 days past their `emission.endDate`, as of 5.4.0
 
 State | Description
 -- | --
@@ -309,6 +310,7 @@ To understand the difference between a task requester and a task owner, kindly s
 * one per analytics reporting period
 * updated when the user loads the application or when they view the targets tab 
 * updated a maximum of once per day
+* [archived automatically](/technical-overview/data/performance/archiving/#automatic-archiving) once their reporting period is more than 6 months old, as of 5.4.0
 
 ```json
 {

content/en/technical-overview/data/performance/archiving.md

@@ -46,7 +46,11 @@ Archiving will cause CouchDB to become fragmented until  [compaction](https://do
 
 ## Queueing documents for archiving
 
-Determining _which_ documents to archive is currently a manual process. For example, you can use analytics queries to find reports older than a retention threshold, or contacts that have been inactive for years.
+{{< callout type="info" >}}
+Expired tasks and targets are [archived automatically](#automatic-archiving).
+{{< /callout >}}
+
+Determining _which_ documents to archive via the API is a manual process. For example, you can use analytics queries to find reports older than a retention threshold, or contacts that have been inactive for years.
 
 A good place to start when considering which documents to archive is to evaluate your current purge configuration and see which documents are already marked as purged for all user roles.
 
@@ -85,6 +89,22 @@ The endpoint splits the IDs into archive jobs and responds with `202 Accepted` a
 
 Jobs are persisted while the payload streams in, so a mid-payload failure can leave some jobs queued even though the request returns an error. Retrying the full payload is safe because archiving is idempotent; duplicate jobs only cost redundant processing.
 
+## Automatic archiving
+
+_Added in `5.4.0`_
+
+Task and target documents expire on their own, and Sentinel archives them without any manual queueing:
+
+| document | archived when                                                                                                                        |
+|----------|--------------------------------------------------------------------------------------------------------------------------------------|
+| Tasks    | The task is in a terminal state (`Cancelled`, `Completed`, or `Failed`) and its `emission.endDate` is more than 60 days in the past. |
+| Targets  | The target's reporting period is more than 6 months in the past.                                                                     |
+
+
+{{< callout type="info" >}}
+Before `5.4.0`, the same documents were [purged](/technical-overview/data/performance/purging/) instead. Archiving replaces that purge and removes the documents from `medic` as well as from user devices.
+{{< /callout >}}
+
 ## Replication
 
 Archived documents appear as deleted to syncing clients. Users who have an archived document on their device delete it during their next sync, and initial replication never downloads archived documents. Users are not notified that documents were archived, so as with purging, the expectation is that documents are only archived once they are no longer relevant to users.
@@ -149,6 +169,7 @@ Each archive job writes a log document to the `medic-logs` database with the sam
 | `start_date` | Number | Timestamp of when processing started |
 | `updated_date` | Number | Timestamp of the last progress update |
 | `errors` | Array | The last 10 errors, each with a `date` and `message` |
+| `automatic` | Boolean | `true` for jobs created by [automatic archiving](#automatic-archiving). Absent on jobs submitted through the API. |
 
 A job whose log has `status: "failed"` has exhausted its 10 attempts and was removed from the queue. Review the `errors` entries, resolve the underlying problem, and resubmit the remaining IDs to the archive endpoint.
 
@@ -174,7 +195,7 @@ On their next sync, users with access to the document download it again with its
 
 ## Server-side processing
 
-Sentinel processes queued jobs in creation order, working through each job's IDs in batches of {{< format-number 1_000 >}}. For each batch, Sentinel:
+Sentinel processes queued jobs in creation order, working through each job's IDs in batches of {{< format-number 1_000 >}}. The [automatic archiving](#automatic-archiving) will only run when there are no other jobs to process. For each batch, Sentinel:
 
 1. Copies the documents, including attachments, to the `medic-archive` database. An `archive_date` timestamp is added to each archived document.
 2. Records an audit entry for each archived document.

content/en/technical-overview/data/performance/purging.md

@@ -6,6 +6,7 @@ description: >
   Remove unneeded documents from offline users devices
 keywords:
 relatedContent: >
+  technical-overview/data/performance/archiving
   design/personas/chw-janet
   building/reports
   community/contributing/code/troubleshooting/invalid-reports
@@ -122,8 +123,7 @@ And should return an array of `_id` values for docs you would like to be purged
 
 In the cases of reports that do not have subjects or their subjects are not found, the `purge` function will receive an empty object as `contact`. In the cases of reports about deleted contacts, the `purge` function will receive a `{ _deleted: true }` object as the `contact`.
 
-As of **3.9.0**, `task` documents that are in a terminal state (`Cancelled`, `Completed`, `Failed`) are purged if their `end_date` is more than 60 days ago (relative to server date).
-As of **3.9.0**, `target` documents are purged if their reporting period is more than 6 months ago (relative to server date). Purging `task` and `target` documents happens automatically on every purge run. The intervals and required states are not configurable.
+From **3.9.0** to **5.3.x**, every purge run also purges `task` documents in a terminal state (`Cancelled`, `Completed`, `Failed`) whose `end_date` is more than 60 days ago, and `target` documents whose reporting period is more than 6 months ago (both relative to server date). As of **5.4.0**, Sentinel [archives these documents automatically](/technical-overview/data/performance/archiving/#automatic-archiving) instead, which removes them from user devices and from the `medic` database.
 
 #### CHT API