-
Notifications
You must be signed in to change notification settings - Fork 1
Feature/8604 add aggregated download job status #335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
utas-raymondng
merged 4 commits into
main
from
feature/8604-add-aggregated-download-job-status
Aug 26, 2026
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
68dc6a4
feat: add aggregated download job status endpoint
grojeda b50963f
Merge branch 'main' of github.com:aodn/ogcapi-java into feature/8604-…
grojeda f6f3b9d
refactor: keep AODN download status fields outside upstream OGC schema
grojeda bd91c65
Merge branch 'main' into feature/8604-add-aggregated-download-job-status
grojeda File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7 changes: 7 additions & 0 deletions
7
.../src/main/java/au/org/aodn/ogcapi/server/core/exception/DownloadJobNotFoundException.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| package au.org.aodn.ogcapi.server.core.exception; | ||
|
|
||
| public class DownloadJobNotFoundException extends RuntimeException { | ||
| public DownloadJobNotFoundException() { | ||
| super("Download job not found"); | ||
| } | ||
| } |
11 changes: 11 additions & 0 deletions
11
...er/src/main/java/au/org/aodn/ogcapi/server/core/exception/DownloadJobStatusException.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| package au.org.aodn.ogcapi.server.core.exception; | ||
|
|
||
| public class DownloadJobStatusException extends RuntimeException { | ||
| public DownloadJobStatusException() { | ||
| super("Unable to retrieve download job status"); | ||
| } | ||
|
|
||
| public DownloadJobStatusException(Throwable cause) { | ||
| super("Unable to retrieve download job status", cause); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
13 changes: 13 additions & 0 deletions
13
server/src/main/java/au/org/aodn/ogcapi/server/core/model/DownloadExecutionResponse.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| package au.org.aodn.ogcapi.server.core.model; | ||
|
|
||
| import au.org.aodn.ogcapi.processes.model.InlineResponse200; | ||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
| import io.swagger.v3.oas.annotations.media.Schema; | ||
|
|
||
| @Schema(description = "Compatible download execution response with the submitted AWS Batch job ID.") | ||
| public record DownloadExecutionResponse( | ||
| @JsonProperty("message") InlineValue message, | ||
| @JsonProperty("status") InlineValue status, | ||
| @JsonProperty("jobID") String jobId | ||
| ) implements InlineResponse200 { | ||
| } | ||
65 changes: 65 additions & 0 deletions
65
server/src/main/java/au/org/aodn/ogcapi/server/core/model/DownloadJobStatusInfo.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| package au.org.aodn.ogcapi.server.core.model; | ||
|
|
||
| import au.org.aodn.ogcapi.processes.model.StatusInfo; | ||
| import com.fasterxml.jackson.annotation.JsonInclude; | ||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
| import io.swagger.v3.oas.annotations.media.Schema; | ||
|
|
||
| /** | ||
| * AODN display metadata added to the standard OGC job status response. | ||
| */ | ||
| @Schema(description = "AODN download job status, extending the standard OGC StatusInfo model.") | ||
| public class DownloadJobStatusInfo extends StatusInfo { | ||
|
|
||
| @JsonProperty("collection") | ||
| @JsonInclude(JsonInclude.Include.NON_ABSENT) | ||
| @Schema(description = "Display name of the requested collection.") | ||
| private String collection; | ||
|
|
||
| @JsonProperty("dataSelection") | ||
| @JsonInclude(JsonInclude.Include.NON_ABSENT) | ||
| @Schema(description = "Dataset key or data selection requested for the download.") | ||
| private String dataSelection; | ||
|
|
||
| @JsonProperty("format") | ||
| @JsonInclude(JsonInclude.Include.NON_ABSENT) | ||
| @Schema(description = "Requested output document format.") | ||
| private String format; | ||
|
|
||
| @JsonProperty("metadataUrl") | ||
| @JsonInclude(JsonInclude.Include.NON_ABSENT) | ||
| @Schema(description = "Link to the metadata page supplied when the download was submitted.") | ||
| private String metadataUrl; | ||
|
|
||
| public String getCollection() { | ||
| return collection; | ||
| } | ||
|
|
||
| public void setCollection(String collection) { | ||
| this.collection = collection; | ||
| } | ||
|
|
||
| public String getDataSelection() { | ||
| return dataSelection; | ||
| } | ||
|
|
||
| public void setDataSelection(String dataSelection) { | ||
| this.dataSelection = dataSelection; | ||
| } | ||
|
|
||
| public String getFormat() { | ||
| return format; | ||
| } | ||
|
|
||
| public void setFormat(String format) { | ||
| this.format = format; | ||
| } | ||
|
|
||
| public String getMetadataUrl() { | ||
| return metadataUrl; | ||
| } | ||
|
|
||
| public void setMetadataUrl(String metadataUrl) { | ||
| this.metadataUrl = metadataUrl; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
16 changes: 16 additions & 0 deletions
16
server/src/main/java/au/org/aodn/ogcapi/server/processes/BatchJobProperties.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| package au.org.aodn.ogcapi.server.processes; | ||
|
|
||
| import org.springframework.boot.context.properties.ConfigurationProperties; | ||
|
|
||
| @ConfigurationProperties(prefix = "aws.batch.job") | ||
| public record BatchJobProperties( | ||
| String queue, | ||
| String definition, | ||
| String childQueue | ||
| ) { | ||
| public BatchJobProperties { | ||
| if (childQueue == null || childQueue.isBlank()) { | ||
| childQueue = queue; | ||
| } | ||
| } | ||
| } |
76 changes: 76 additions & 0 deletions
76
server/src/main/java/au/org/aodn/ogcapi/server/processes/DownloadJobStatusAggregator.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| package au.org.aodn.ogcapi.server.processes; | ||
|
|
||
| import au.org.aodn.ogcapi.processes.model.StatusCode; | ||
| import software.amazon.awssdk.services.batch.model.JobStatus; | ||
|
|
||
| import java.util.Objects; | ||
| import java.util.stream.Stream; | ||
|
|
||
| /** | ||
| * Maps an already-fetched AWS Batch workflow snapshot to its public OGC status. | ||
| * This class deliberately makes no AWS calls so the workflow rules remain deterministic. | ||
| */ | ||
| public class DownloadJobStatusAggregator { | ||
|
|
||
| public enum WorkflowMode { | ||
| EXPLICIT_ZARR, | ||
| CHILD_DISCOVERY_REQUIRED | ||
| } | ||
|
|
||
| public record Snapshot( | ||
| JobStatus initial, | ||
| JobStatus prepare, | ||
| JobStatus collect, | ||
| WorkflowMode workflowMode, | ||
| boolean discoveryWindowExpired | ||
| ) { | ||
| public Snapshot { | ||
| Objects.requireNonNull(initial, "initial status is required"); | ||
| Objects.requireNonNull(workflowMode, "workflow mode is required"); | ||
| } | ||
| } | ||
|
|
||
| public StatusCode aggregate(Snapshot snapshot) { | ||
| if (Stream.of(snapshot.initial(), snapshot.prepare(), snapshot.collect()) | ||
| .anyMatch(status -> status == JobStatus.FAILED)) { | ||
| return StatusCode.FAILED; | ||
| } | ||
|
|
||
| if (Stream.of(snapshot.initial(), snapshot.prepare(), snapshot.collect()) | ||
| .filter(Objects::nonNull) | ||
| .anyMatch(status -> status == JobStatus.UNKNOWN_TO_SDK_VERSION)) { | ||
| throw new IllegalStateException("Unsupported AWS Batch job status"); | ||
| } | ||
|
|
||
| boolean hasPrepare = snapshot.prepare() != null; | ||
| boolean hasCollect = snapshot.collect() != null; | ||
| if (hasPrepare != hasCollect) { | ||
| if (snapshot.discoveryWindowExpired()) { | ||
| throw new IllegalStateException("Only one child workflow job was found after the discovery window"); | ||
| } | ||
| return StatusCode.RUNNING; | ||
| } | ||
|
|
||
| if (snapshot.collect() == JobStatus.SUCCEEDED) { | ||
| return StatusCode.SUCCESSFUL; | ||
| } | ||
|
|
||
| if (hasPrepare) { | ||
| return StatusCode.RUNNING; | ||
| } | ||
|
|
||
| return switch (snapshot.initial()) { | ||
| case SUBMITTED, PENDING, RUNNABLE -> StatusCode.ACCEPTED; | ||
| case STARTING, RUNNING -> StatusCode.RUNNING; | ||
| case SUCCEEDED -> { | ||
| if (snapshot.workflowMode() == WorkflowMode.EXPLICIT_ZARR | ||
| || snapshot.discoveryWindowExpired()) { | ||
| yield StatusCode.SUCCESSFUL; | ||
| } | ||
| yield StatusCode.RUNNING; | ||
| } | ||
| case FAILED -> StatusCode.FAILED; | ||
| default -> throw new IllegalStateException("Unsupported AWS Batch job status"); | ||
| }; | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.