-
Notifications
You must be signed in to change notification settings - Fork 588
HDDS-14123. Refactor BucketEndpoint#Put method #9516
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @echonesis for working on this. I would like to simplify the implementation, but first need to do some refactoring in S3G to make it easier. (#9519 done, #9522 PR ready, HDDS-14204 in progress, HDDS-14198, maybe more)
| * Context object that provides access to BucketEndpoint resources. | ||
| * This allows handlers to access endpoint functionality without | ||
| * tight coupling to the BucketEndpoint class. | ||
| * | ||
| * Since BucketEndpoint extends EndpointBase, handlers can access: | ||
| * - Bucket and Volume operations | ||
| * - Methods inherited from EndpointBase | ||
| */ | ||
| public class BucketEndpointContext { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we should introduce BucketEndpointContext. The context duplicates "random" functionality of BucketEndpoint. It would likely need to grow when extracting other PUT operations (tagging, etc.), GET operations. We will also want to extract handlers from ObjectEndpoint.
Handlers can be considered specialised "mini" endpoints. Thus they should simply extend EndpointBase to inherit all required functionality (configuration, headers, request context, audit logging, etc.).
| Map<String, String> queryParams = new HashMap<>(); | ||
| queryParams.put("acl", aclMarker); | ||
| // Future handlers: queryParams.put("lifecycle", lifecycleMarker); | ||
|
|
||
| // Check for subresource operations using handlers | ||
| String queryParam = HANDLER_FACTORY.findFirstSupportedQueryParam(queryParams); | ||
|
|
||
| if (queryParam != null) { | ||
| BucketOperationHandler handler = HANDLER_FACTORY.getHandler(queryParam); | ||
| // Delegate to specific handler | ||
| s3GAction = getActionForQueryParam(queryParam); | ||
| Response response = handler.handlePutRequest( | ||
| bucketName, body, headers, getContext(), startNanos); | ||
| AUDIT.logWriteSuccess( | ||
| buildAuditMessageForSuccess(s3GAction, getAuditParameters())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is too complex.
- Query params are already available in a map via
ContainerRequestContext. - Handlers should take care of audit logging and metrics. The logic to decide value of
s3GActionshould not be in centralized inBucketEndpoint.
It is better to let handlers inspect query params (and possibly other inputs, like headers) to decide if they should handle the request or not.
Something like (in AclHandler):
private boolean shouldHandle() {
return hasQueryParam(ACL);
}
public Response handlePutRequest(String bucketName, InputStream body) ... {
if (!shouldHandle()) {
return null;
}
... handle bucket ACL putThen we can simply try all handlers until one of them does the work:
for (BucketOperationHandler handler : handlers) {
Response response = handler.handlePutRequest(bucketName, body);
if (response != null) {
return response;
}
}
... continue handling in `BucketEndpoint`|
Thanks @adoroszlai for the review. |
adoroszlai
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @echonesis for updating the patch (and sorry for the late review after holidays).
| * This handler extends EndpointBase to inherit all required functionality | ||
| * (configuration, headers, request context, audit logging, metrics, etc.). | ||
| */ | ||
| public class AclHandler extends EndpointBase implements BucketOperationHandler { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please rename to BucketAclHandler. Object ACL is not implemented yet, but is planned (HDDS-9631).
| private List<OzoneAcl> getAndConvertAclOnBucket(String value, | ||
| String permission) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Please do not format method signature like this. Whenever visibility / return type / method name / other modifiers are changed, we would have to reindent all parameters.
I know it's the same as it was in BucketEndpoint, but we can improve formatting when moving code to a new file.
(Applies to few other similar methods.)
| private boolean listKeysShallowEnabled; | ||
| private int maxKeysLimit = 1000; | ||
|
|
||
| private List<BucketOperationHandler> putHandlers; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BucketOperationHandler will eventually handle other operations (get, etc.), so this should be simply handlers.
| Response handlePutRequest(String bucketName, InputStream body) | ||
| throws IOException, OS3Exception; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some handlers may not implement PUT, so let's make this a default method that return null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing it out!
I will update it in the next commit.
| ozoneAclList.add(OzoneAcl.of( | ||
| IAccessAuthorizer.ACLIdentityType.USER, | ||
| userId, | ||
| OzoneAcl.AclScope.DEFAULT, | ||
| aclsOnBucket | ||
| )); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: please import static for USER and DEFAULT, then this can be written in a single line.
(Also two other similar statements below.)
| // Create a spy of AclHandler to verify audit logging | ||
| AclHandler spyHandler = spy(EndpointBuilder.newAclHandlerBuilder() | ||
| .setClient(client) | ||
| .setHeaders(headers) | ||
| .build()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can spy(aclHandler), no need to create new instance.
| // Set up query parameter to indicate ACL operation | ||
| aclHandler.queryParamsForTest().set("acl", ""); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be added in setup() to reduce duplication, then unset only in testHandlePutRequestWithoutAclQueryParam.
| Response response = aclHandler.handlePutRequest(BUCKET_NAME, null); | ||
|
|
||
| assertEquals(HTTP_OK, response.getStatus(), | ||
| "PUT ACL should return 200 OK"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can simplify a bit:
assertSucceeds(() -> aclHandler.handlePutRequest(BUCKET_NAME, null));| when(headers.getHeaderString(S3Acl.GRANT_WRITE)) | ||
| .thenReturn("id=\"testuser\""); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
testHandlePutRequestWithReadHeader, testHandlePutRequestWithWriteHeader, testHandlePutRequestWithReadAcpHeader, testHandlePutRequestWithWriteAcpHeader, testHandlePutRequestWithFullControlHeader can be combined into a single @ParameterizedTest.
| OS3Exception exception = assertThrows(OS3Exception.class, () -> { | ||
| aclHandler.handlePutRequest(BUCKET_NAME, null); | ||
| }, "Should throw OS3Exception for unsupported grantee type"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: {} is unnecessary (in all assertThrows calls).
| OS3Exception exception = assertThrows(OS3Exception.class, () -> { | ||
| aclHandler.handlePutRequest(BUCKET_NAME, null); | ||
| }, "Should throw OS3Exception for email address grantee type"); | ||
|
|
||
| assertEquals(HTTP_NOT_IMPLEMENTED, exception.getHttpCode(), | ||
| "Should return NOT_IMPLEMENTED for email address grantee type"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These can be simplified to:
assertErrorResponse(S3ErrorTable.NOT_IMPLEMENTED,
() -> aclHandler.handlePutRequest(BUCKET_NAME, null));
adoroszlai
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @echonesis for updating the patch.
|
Thanks @adoroszlai for the review and merge. |
What changes were proposed in this pull request?
Refactored the BucketEndpoint#put method by extracting ACL handling logic into dedicated handler classes, improving code modularity and maintainability through the Strategy pattern.
Changes
AclHandlerclass to handle bucket ACL PUT operationsBucketOperationHandlerinterface for pluggable operation handlersBucketEndpoint#put()to delegate ACL operations toAclHandlerTesting
TestAclHandlerTestBucketAcltestsWhat is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14123
How was this patch tested?
GitHub Actions CI: https://github.com/echonesis/ozone/actions/runs/20752426565