Attachment upload can block pytest indefinitely because the configured API timeout is not applied

[ArtyrKa1300]

hello!

Description

We have reproduced an intermittent hang in qase-pytest when uploading attachments.

The tests finish normally, but one pytest-xdist worker remains active indefinitely. Pytest cannot complete and is eventually terminated by the CI timeout.

The issue occurs when an attachment upload starts but the HTTP request neither returns a response nor raises an exception.

Environment

• qase-pytest==9.0.0
• qase-python-commons==5.1.4
• qase-api-client==2.0.14
• qase-api-v2-client==2.0.8
• pytest==9.0.3
• pytest-xdist==3.8.0
• urllib3==2.7.0
• Python 3.13.14
• Four xdist workers
• Result batch size: 20 - 50
• Custom Qase API host/proxy

Configuration

QASE_TESTOPS_API_TIMEOUT=30
QASE_TESTOPS_API_RETRIES=3
QASE_TESTOPS_API_RETRY_BACKOFF=2

Actual behavior

The Qase debug log contains:

• 2,519 attachment upload starts
• 2,518 successful attachment uploads
• No error for the remaining attachment upload
• Three completed Qase workers out of four
• No Qase run completion event

The unmatched operation is:

[Qase][debug] Uploading batch 1/1 with 6 file(s) for project AA

There is no corresponding message:

Successfully uploaded batch ...

or:

Error uploading batch ...

The affected reporter thread remains active indefinitely.

The configured result batch size was 20. A total of 2,430 results were created, but only 2,410 reached the API v2 result submission stage. Therefore, the complete batch containing the stalled attachment upload was never submitted.

API v2 retries work correctly

During the same run, five API v2 result submission requests failed with:

ConnectionResetError: Connection reset by peer

All five requests were retried successfully.

There were:

• 128 API v2 submission attempts
• 5 retry events
• 123 successful logical batch submissions
• No failures after retry exhaustion

The hang is therefore related to attachment upload through the API v1 client, rather than API v2 result submission.

Technical analysis

ApiV2Client.send_results() prepares all results before submitting the batch:

results_to_send = [
    self._prepare_result(project_code, result)
    for result in results
]

During result preparation, attachments are uploaded through _upload_attachment():

attach_id = self._upload_attachment(
    project_code,
    attachments_to_upload
)

The attachment request is made through the API v1 client:

response = attach_api.upload_attachment(
    project_code,
    file=files_for_upload
)

The generated AttachmentsApi.upload_attachment() method supports the _request_timeout argument, but qase-python-commons does not pass it.

The configured API timeout is passed to API v2 result submission:

api_results.create_results_v2(
    ...,
    _request_timeout=self.config.testops.api.timeout
)

However, it is not passed to API v1 attachment uploads.

If the attachment connection remains open without returning data, the call blocks indefinitely. Since no exception is raised, the outer send_with_retry() wrapper cannot perform a retry.

The reporter thread remains active, and complete_worker() waits indefinitely:

while self.count_running_threads > 0:
    time.sleep(DEFAULT_THREAD_POLL_INTERVAL)

Consequently, the xdist worker and the entire pytest process cannot finish.

There is also a related retry issue: _upload_attachment() catches attachment exceptions internally and continues. Therefore, attachment failures that do raise an exception are not handled by the outer retry mechanism.

Expected behavior

Attachment uploads should:

• Use the configured API timeout
• Have bounded retry behavior for retryable transport failures
• Never block a reporter thread indefinitely
• Allow results to be submitted even if an attachment cannot be uploaded
• Allow all pytest-xdist workers to exit normally

Suggested fix

Pass the configured timeout to the attachment API call:

response = attach_api.upload_attachment(
    project_code,
    file=files_for_upload,
    _request_timeout=self.config.testops.api.timeout
)

Attachment uploads should also have a separate bounded retry mechanism.

After attachment retry exhaustion, the reporter should log the failure, omit the affected attachment, and continue submitting the result batch.

Acceptance criteria

• QASE_TESTOPS_API_TIMEOUT is applied to API v1 attachment uploads
• A stalled attachment upload is interrupted after the configured timeout
• Retryable attachment failures are retried a bounded number of times
• Attachment retry exhaustion does not prevent result submission
• A reporter thread cannot remain active indefinitely because of an attachment request
• All pytest-xdist workers exit after an attachment timeout

[gibiw]

Hi @ArtyrKa1300, thanks for the very thorough report — the analysis and the log counts made this one straightforward to confirm.

Fixed in qase-python-commons 5.1.5, now on PyPI. Your diagnosis was correct on every point:

• The configured testops.api.timeout reached API v2 result submission but never the API v1 client, so a stalled attachment upload was waited on indefinitely. Every API v1 call now passes _request_timeout — attachment uploads plus get_project, get_environments, get_configurations, create_configuration_group, create_configuration, get_run, complete_run, create_run, update_run_publicity and run_update_external_issue. The ones on the main thread had the same unbounded-wait problem.
• Attachment batches are now retried through the same bounded policy as result uploads (testops.api.retries / testops.api.retryBackoff): transport failures and HTTP 408/429/5xx are retried, 4xx are not. Previously the per-batch except swallowed attachment errors before the outer send_with_retry() could see them.
• After the attempts are exhausted the reporter logs the failure, drops those attachments and still submits the results they belong to — a failed attachment can no longer cost you the whole batch. Remaining batches keep uploading.

PR: #514

Your QASE_TESTOPS_API_TIMEOUT=30 / QASE_TESTOPS_API_RETRIES=3 / QASE_TESTOPS_API_RETRY_BACKOFF=2 settings need no changes — they now apply to attachments as well.

pip install -U qase-python-commons==5.1.5

qase-pytest pins qase-python-commons~=5.1.2, so 5.1.5 is picked up without a reporter upgrade.

Could you try it on your CI and let us know whether the hang is gone? We're leaving this issue open until you confirm. If a worker still stalls, the debug log around Uploading batch ... would tell us whether a request now times out and retries as expected.

One note on your last remark: the repeated Error preparing attachment ... text/plain messages are untouched by this change, as you expected — they come from get_for_upload() and are unrelated to the hang.

[ArtyrKa1300]

@gibiw Hi, thank you for the quick fix!

We have been running qase-python-commons==5.1.5 in our CI for several days, and everything has been stable. We have not observed any further stalled workers or CI runs hanging during attachment uploads. Test results are delivered successfully, and Qase runs complete normally.

The issue appears to be fully resolved on our side and can be closed.

Thanks again!