Skip to content

ClusterSummary status does not surface removeResourceSummary failures #1945

Description

@gianlucam76

What happens

ClusterSummaryReconciler.prepareForDeployment (controllers/clustersummary_controller.go) calls removeResourceSummary before deploying. When that call fails, the error is logged and the reconcile is requeued, but ClusterSummary.status is left completely untouched:

if !clusterSummaryScope.IsContinuousWithDriftDetection() {
    if r.anyFeatureNeedsResourceSummaryRemoval(clusterSummaryScope) {
        err = r.removeResourceSummary(ctx, clusterSummaryScope, logger)
        if err != nil {
            logger.V(logs.LogInfo).Error(err, "failed to remove ResourceSummary.")
            r.setNextReconcileTime(clusterSummaryScope, normalRequeueAfter)
            return reconcile.Result{RequeueAfter: normalRequeueAfter}
        }
        r.markResourceSummaryRemovedForAllFeatures(clusterSummaryScope)
    }
}

The same pattern exists in the delete path, cleanupBeforeFinalizerRemoval (same file):

err = r.removeResourceSummary(ctx, clusterSummaryScope, logger)
if err != nil {
    logger.V(logs.LogInfo).Error(err, "failed to remove ResourceSummary.")
    return reconcile.Result{Requeue: true, RequeueAfter: deleteRequeueAfter}, nil, true
}

In both cases ClusterSummary.status.featureSummaries keeps whatever value it last had (often Provisioning, or nothing at all on a fresh ClusterSummary), and no failureMessage is ever set. An ongoing failure — including a real connectivity or auth problem with the managed cluster — is invisible to kubectl get clustersummary; it only shows up in controller logs.

Contrast with updateChartMap's handling of a missing required TemplateResourceRef, three lines above the first snippet:

err = r.updateChartMap(ctx, clusterSummaryScope, logger)
if err != nil {
    if apierrors.IsNotFound(err) {
        r.setFailureMessage(clusterSummaryScope, err.Error())
        r.resetFeatureStatus(clusterSummaryScope, libsveltosv1beta1.FeatureStatusFailedNonRetriable)
    }
    ...
}

That path does surface the error into status via the existing setFailureMessage/resetFeatureStatus helpers.

How this was found

Testing the new generic OIDC workload identity provider (libsveltos WorkloadIdentityProviderOIDC) against a real managed cluster: a TLS trust failure talking to the IdP kept removeResourceSummary failing every reconcile, logged repeatedly as "failed to remove ResourceSummary.", while ClusterSummary.status sat at:

status:
  dependencies: no dependencies
  featureSummaries:
  - featureID: Resources
    hash: /BrkVtt9bmqS4wvPD9/9J1TGRFd2sBsK0XSHwvJGshI=
    status: Provisioning

with no indication anything was wrong. The same gap would show up for any transient or persistent connectivity/auth failure, regardless of provider (kubeconfig, cloud workload identity, or OIDC).

Suggested fix

Surface the error from removeResourceSummary into ClusterSummary.status using the existing setFailureMessage/resetFeatureStatus helpers, the same way updateChartMap's error handling already does, in both call sites (prepareForDeployment and cleanupBeforeFinalizerRemoval). Since this class of failure (connectivity/auth) is generally transient and already retried via requeue, FeatureStatusFailed (retriable) looks like the right status rather than FeatureStatusFailedNonRetriable.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions