HDDS-15809. Validate x-amz-copy-source-range on UploadPartCopy by rich7420 · Pull Request #10711 · apache/ozone

rich7420 · 2026-07-10T11:33:21Z

What changes were proposed in this pull request?

UploadPartCopy read the x-amz-copy-source-range header but never validated it
against the source object. In ObjectEndpoint.createMultipartKey the range was
parsed with a hard-coded length of 0 and the result was never checked, so:

an out-of-range range was silently clamped and surfaced as a generic
InvalidRequest, and
a malformed range did not match the parser's pattern and fell through to
"read the whole object", raising no error at all.

This aligns UploadPartCopy with AWS S3 by validating the header:

If the value is not a single numeric byte range (bytes=<start>-<end>), return
InvalidArgument (HTTP 400). This rejects values such as 0-2 (missing
bytes=), bytes=0 (no end), bytes=hello-world, bytes=0-bar, bytes=hello-,
and bytes=0-2,3-5 (multiple ranges).
If start > end or end is beyond the source object size, return
InvalidRange (HTTP 416; AWS also permits 400). Example: a 5-byte source with
bytes=0-21.
A valid range is unchanged: the target part length is end - start + 1 and the
existing copy path is used.

Both InvalidArgument and InvalidRange already exist in S3ErrorTable, so no new
error codes or wire changes are needed. The change is confined to the UploadPartCopy
branch; CopyObject (without a part) and normal GetObject/PutObject paths are
untouched, and the shared RangeHeaderParserUtil (used by GetObject) is not
modified.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15809

How was this patch tested?

https://github.com/rich7420/ozone/actions/runs/29078906089

The UploadPartCopy path read x-amz-copy-source-range but never validated it against the source object: the range was parsed with a hard-coded length of 0 and the result was never checked, so an out-of-range range was silently clamped (surfacing as InvalidRequest) and a malformed range fell through to reading the whole object (no error at all). Validate the header in ObjectEndpoint.createMultipartKey: reject a value that is not a single numeric byte range (bytes=<start>-<end>) with InvalidArgument, and reject a range whose start > end or whose end is beyond the source size with InvalidRange. Add an AWS SDK v2 integration test covering both cases.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

adoroszlai

Thanks @rich7420 for the patch.

adoroszlai · 2026-07-10T12:50:57Z

+  private static final Pattern COPY_SOURCE_RANGE_PATTERN =
+      Pattern.compile("^bytes=(\\d+)-(\\d+)$");


Duplicate?

ozone/hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3Consts.java

Lines 68 to 69 in cd8ba56

public static final Pattern RANGE_HEADER_MATCH_PATTERN =

Pattern.compile("bytes=(?<start>[0-9]*)-(?<end>[0-9]*)");

adoroszlai · 2026-07-10T12:56:56Z

+          Matcher matcher = COPY_SOURCE_RANGE_PATTERN.matcher(range);
+          if (!matcher.matches()) {
+            throw newError(S3ErrorTable.INVALID_ARGUMENT, range);
+          }
+          long startOffset = Long.parseLong(matcher.group(1));
+          long endOffset = Long.parseLong(matcher.group(2));
+          long sourceSize = sourceKeyDetails.getDataSize();
+          if (startOffset > endOffset || endOffset >= sourceSize) {
+            throw newError(S3ErrorTable.INVALID_RANGE, range);
+          }
+          rangeHeader = new RangeHeader(startOffset, endOffset, false, false);


Can RangeHeaderParserUtil can be tweaked to make the range mandatory in this case?

adoroszlai · 2026-07-10T12:59:09Z

+    UploadPartCopyRequest outOfRangeRequest = UploadPartCopyRequest.builder()
+        .sourceBucket(sourceBucketName)
+        .sourceKey(sourceKey)
+        .destinationBucket(destBucketName)
+        .destinationKey(destKey)
+        .uploadId(uploadId)
+        .partNumber(1)


Can we store Builder instance and use it to build all requests, where only copySourceRange is different?

adoroszlai · 2026-07-10T13:05:00Z

+        .build();
+    S3Exception outOfRange = assertThrows(S3Exception.class, () -> s3Client.uploadPartCopy(outOfRangeRequest));
+    // InvalidRange maps to HTTP 416; AWS also permits 400 for this case.
+    assertTrue(outOfRange.statusCode() == 400 || outOfRange.statusCode() == 416,


nit: please use assertThat instead of assertTrue (HDDS-9951), and use HttpStatus constants.

assertThat(outOfRange.statusCode()).isIn(SC_BAD_REQUEST, SC_REQUESTED_RANGE_NOT_SATISFIABLE);

rich7420 · 2026-07-11T05:52:58Z

@adoroszlai thanks for the review, I'll try to fix it up

- Reuse S3Consts.RANGE_HEADER_MATCH_PATTERN for the copy-source-range check instead of adding a duplicate pattern; reject empty start/end as InvalidArgument. - In testUploadPartCopyInvalidRange, share one UploadPartCopyRequest.Builder and vary only copySourceRange. - Use assertThat with HttpStatus constants (HDDS-9951) instead of assertTrue with magic numbers.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The v2 SDK test uses org.apache.http.HttpStatus (SC_BAD_REQUEST / SC_REQUESTED_RANGE_NOT_SATISFIABLE); declare httpcore as a test dependency so maven-dependency-plugin analyze does not fail with a used-undeclared dependency.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

adoroszlai

Thanks @rich7420 for updating the patch.

Copilot AI review requested due to automatic review settings July 10, 2026 11:33

Copilot AI reviewed Jul 10, 2026

adoroszlai reviewed Jul 10, 2026

View reviewed changes

adoroszlai added the s3 S3 Gateway label Jul 10, 2026

Copilot AI review requested due to automatic review settings July 11, 2026 06:42

Copilot AI reviewed Jul 11, 2026

Copilot AI review requested due to automatic review settings July 11, 2026 09:38

Copilot AI reviewed Jul 11, 2026

adoroszlai requested review from Gargi-jais11, ivandika3 and peterxcli July 11, 2026 11:48

adoroszlai reviewed Jul 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-15809. Validate x-amz-copy-source-range on UploadPartCopy#10711

HDDS-15809. Validate x-amz-copy-source-range on UploadPartCopy#10711
rich7420 wants to merge 3 commits into
apache:masterfrom
rich7420:HDDS-15809

rich7420 commented Jul 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

adoroszlai left a comment

Uh oh!

adoroszlai Jul 10, 2026

Uh oh!

adoroszlai Jul 10, 2026

Uh oh!

adoroszlai Jul 10, 2026

Uh oh!

adoroszlai Jul 10, 2026

Uh oh!

rich7420 commented Jul 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

adoroszlai left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		private static final Pattern COPY_SOURCE_RANGE_PATTERN =
		Pattern.compile("^bytes=(\\d+)-(\\d+)$");

	public static final Pattern RANGE_HEADER_MATCH_PATTERN =
	Pattern.compile("bytes=(?<start>[0-9])-(?<end>[0-9])");

Uh oh!

Conversation

rich7420 commented Jul 10, 2026

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

adoroszlai Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

adoroszlai Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

adoroszlai Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

adoroszlai Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

rich7420 commented Jul 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants