limit concurrent downloads per user and queue the rest

[grojeda]

A user (identified by their recipient email) can now have at most 10 downloads in flight at once; anything beyond that is queued inside ogcapi and submitted to AWS Batch automatically as their slots free, rather than rejected — a queued download still gets a job ID and answers on the status endpoint. A slot stays occupied for the whole workflow (the master job plus the prepare/collect children DAS spawns), counted from one cached sweep of the Batch queues shared by all users. The execute and status responses gain queued and queuePosition so the portal can show the queued state without parsing message text; both additions are backwards compatible. Configurable under aws.batch.job.user-limit — max-concurrent defaults to 10, enabled: false turns it off. Note the recipient email is not authenticated, so this is a fairness mechanism rather than a security control.

[Copilot]

Pull request overview

Adds per-recipient download concurrency limits, queueing excess requests locally and releasing them as AWS Batch capacity becomes available.

Changes:

• Adds cached AWS Batch workflow counting and queued-download admission.
• Extends execution/status responses with queue state and position.
• Adds configuration and comprehensive unit tests.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
server/src/main/resources/application.yaml	Adds user-limit defaults.
server/src/test/resources/application-test.yaml	Disables admission limits in integration tests.
server/src/main/java/.../core/configuration/Config.java	Registers limit properties.
server/src/main/java/.../core/model/DownloadExecutionResponse.java	Adds execution queue fields.
server/src/main/java/.../core/model/DownloadJobStatusInfo.java	Adds status queue fields.
server/src/main/java/.../processes/DownloadAdmission.java	Models admission results.
server/src/main/java/.../processes/DownloadAdmissionService.java	Implements holding and scheduled release.
server/src/main/java/.../processes/DownloadJobStatusService.java	Resolves held and released jobs.
server/src/main/java/.../processes/DownloadLimitProperties.java	Defines configurable limits.
server/src/main/java/.../processes/DownloadRequest.java	Groups download inputs.
server/src/main/java/.../processes/HeldDownload.java	Models queued downloads.
server/src/main/java/.../processes/InFlightDownloadCounter.java	Counts active AWS workflows.
server/src/main/java/.../processes/RestApi.java	Routes execution through admission.
server/src/main/java/.../processes/RestServices.java	Separates parameter construction and submission.
server/src/test/java/.../processes/DownloadAdmissionServiceTest.java	Tests admission and release behavior.
server/src/test/java/.../processes/DownloadJobStatusServiceTest.java	Tests queued-status resolution.
server/src/test/java/.../processes/InFlightDownloadCounterTest.java	Tests workflow counting.
server/src/test/java/.../processes/MutableTestClock.java	Supports deterministic time tests.
server/src/test/java/.../processes/RestApiJobsTest.java	Tests queue response serialization.
server/src/test/java/.../processes/RestApiTest.java	Tests admission integration.
server/src/test/java/.../processes/RestServicesTest.java	Adapts submission tests to the split API.

Suppressed comments (2)

server/src/main/java/au/org/aodn/ogcapi/server/processes/DownloadAdmissionService.java:261

• This removes the public-to-AWS ID mapping 24 hours after submission, regardless of whether the AWS job has completed. A job that remains pending or running longer than 24 hours will then be looked up using the locally minted public UUID and report not found even though it is still active. Retain mappings until the AWS workflow is terminal, then apply any post-completion retention period.

    private void pruneReleased() {
        Instant cutoff = clock.instant().minus(RELEASED_RETENTION);
        released.entrySet().removeIf(entry -> entry.getValue().releasedAt().isBefore(cutoff));

server/src/main/java/au/org/aodn/ogcapi/server/processes/DownloadAdmissionService.java:253

• An accepted queued download is silently removed once this age is reached, so its job ID switches from accepted to 404 and it is never submitted even if a slot later frees. That contradicts the advertised behavior that excess downloads are queued and submitted automatically. Either keep accepted work until release, or retain an explicit terminal failure/dismissed record that the status endpoint can return instead of losing the job.

        held.removeIf(job -> {
            if (job.acceptedAt().isBefore(cutoff)) {
                heldById.remove(job.jobId());
                log.warn("Abandoning download {} held since {}", job.jobId(), job.acceptedAt());
                return true;

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[weited]

Automated review: flagging the top 3 correctness/concurrency findings (of 14 total) around the new per-user download admission logic.

[weited]

@grojeda
One architecture question worth flagging for offline discussion: the per-user admission logic (DownloadAdmissionService's held queue, plus InFlightDownloadCounter's grace-window bookkeeping) lives entirely in per-instance JVM memory, not shared state. Since ogc api service runs multi-instance and will scale up, that could affect both the accuracy of the per-user cap and the durability of queued downloads across restarts/scaling.
Discuss needed