feat(bigtable): Rerouted MutateRows to use the data client - #18195
feat(bigtable): Rerouted MutateRows to use the data client#18195daniel-sanche wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the mutate_rows method in google/cloud/bigtable/table.py to delegate bulk mutation execution directly to the underlying data client implementation (self._table_impl.bulk_mutate_rows), allowing the removal of the legacy _RetryableMutateRowsWorker class and its associated helper functions. The review feedback correctly identifies a critical bug where an AttributeError will occur if retry is None, and points out the loss of defensive input validation checks for row types and table mismatches that were previously present.
| if timeout is DEFAULT: | ||
| timeout = self.mutation_timeout | ||
|
|
||
| retryable_mutate_rows = _RetryableMutateRowsWorker( | ||
| self._instance._client, | ||
| self.name, | ||
| rows, | ||
| app_profile_id=self._app_profile_id, | ||
| timeout=timeout, | ||
| retryable_errors = RETRYABLE_MUTATION_ERRORS | ||
|
|
||
| # The data client cannot take in zero or null values for deadline, so we set it to | ||
| # the default if that is the case. | ||
| if retry.deadline is None: | ||
| operation_timeout = TABLE_DEFAULT.MUTATE_ROWS | ||
|
|
||
| # To adhere to the retry strategy of do-nothing being achievable with a deadline | ||
| # of 0.0, we modify the retryable errors to be empty if such a deadline is passed. | ||
| elif retry.deadline == 0: | ||
| operation_timeout = TABLE_DEFAULT.MUTATE_ROWS | ||
| retryable_errors = [] | ||
| else: | ||
| operation_timeout = retry.deadline | ||
|
|
||
| attempt_timeout = timeout | ||
| mutation_entries = [ | ||
| RowMutationEntry(row.row_key, row._get_mutations()) for row in rows | ||
| ] | ||
| return_statuses = [status_pb2.Status(code=code_pb2.Code.OK)] * len( | ||
| mutation_entries | ||
| ) # By default, return status OKs for everything |
There was a problem hiding this comment.
Issues Identified:
-
Potential
AttributeErrorwhenretryisNone:
If a user callsmutate_rowswithretry=None(a common pattern to disable retries), accessingretry.deadlineon line 753 will raise anAttributeError: 'NoneType' object has no attribute 'deadline'. We should check ifretry is Nonefirst and disable retries by settingretryable_errors = []. -
Missing Input Validation for Row Type and Table Mismatch:
The previous helper functions_check_row_typeand_check_row_table_namewere removed. Without these checks:- Passing a
ConditionalRoworAppendRowwill raise an unhelpfulAttributeError(due to missing_get_mutations) instead of a clearTypeError. - Passing a row belonging to a different table will silently succeed and write to the current table, potentially causing silent data corruption.
- Passing a
We should restore these defensive checks and handle retry is None gracefully.
if timeout is DEFAULT:
timeout = self.mutation_timeout
retryable_errors = RETRYABLE_MUTATION_ERRORS
# The data client cannot take in zero or null values for deadline, so we set it to
# the default if that is the case.
if retry is None:
operation_timeout = TABLE_DEFAULT.MUTATE_ROWS
retryable_errors = []
elif retry.deadline is None:
operation_timeout = TABLE_DEFAULT.MUTATE_ROWS
# To adhere to the retry strategy of do-nothing being achievable with a deadline
# of 0.0, we modify the retryable errors to be empty if such a deadline is passed.
elif retry.deadline == 0:
operation_timeout = TABLE_DEFAULT.MUTATE_ROWS
retryable_errors = []
else:
operation_timeout = retry.deadline
attempt_timeout = timeout
mutation_entries = []
for row in rows:
if not isinstance(row, DirectRow):
raise TypeError(
"Bulk processing can not be applied for conditional or append mutations."
)
if row.table is not None and row.table.name != self.name:
raise TableMismatchError(
"Row %s is a part of %s table. Current table: %s"
% (row.row_key, row.table.name, self.name)
)
mutation_entries.append(
RowMutationEntry(row.row_key, row._get_mutations())
)
return_statuses = [status_pb2.Status(code=code_pb2.Code.OK)] * len(
mutation_entries
) # By default, return status OKs for everything
Migrating over @gkevinzheng PR from bigtable monorepo googleapis/python-bigtable#1290
Original description:
Note to reviewers: This PR has already been reviewed and merged to a staging branch, with the intention of doing a single merge to main. We are now planning to slowly rollout these changes back to the main branch. Minimal re-review should be necessary