NIFI-16298 Expose Primary Node and Cluster Coordinator role as Prometheus metrics - #11628
NIFI-16298 Expose Primary Node and Cluster Coordinator role as Prometheus metrics#11628abij wants to merge 1 commit into
Conversation
|
@exceptionfactory I was building Grafana dashboards for NiFi Clusters and noticed 2 issues and 1 improvement. First of all so cool that this long-lived project is still so actively maintained. I have created JIRA-tickets and implementation of 3 separate small PR's (with some help of AI, less honor/pride to gain for myself...). Please have a look and have a nice weekend. |
|
Thanks for the contribution @abij - do you mind rebasing your PR as I merged the other one and there are now merge conflicts? |
|
@pvillard31 I have locally resolved the conflict with some struggles and pushed it, it's tested TestFlowResource (22 run, 0 failures). For the Migration guide a suggestion:
|
|
@pvillard31 my commit was not signed, I have resolved this and pushed again. |
|
|
||
| final List<Sample> registryList = metrics.get(ROOT_FIELD_NAME); | ||
| assertEquals(13, registryList.size()); | ||
| assertEquals(15, registryList.size()); |
There was a problem hiding this comment.
Can we assert the names and values of both new metrics instead of only updating the sample counts? The current assertions would pass if either metric were renamed or its boolean value were inverted.
Summary
NIFI-16298 - Expose Primary Node and Cluster Coordinator role as Prometheus metrics
The Prometheus flow metrics endpoint (
/nifi-api/flow/metrics/prometheus) exposes cluster membership/connectivity metrics (cluster_is_clustered,cluster_is_connected_to_cluster,cluster_connected_node_count,cluster_total_node_count), but nothing indicates which node currently holds the Primary Node or Cluster Coordinator role. Building a per-node cluster-overview dashboard currently has no way to mark those roles without querying NiFi's REST API separately and joining the data out-of-band.The data is already available internally and simply wasn't wired into these metrics:
FlowController.isPrimary()andFlowController.isClusterCoordinator()already exist, following the exact same pattern asFlowController.isClustered(), whichControllerFacade/StandardNiFiServiceFacadealready use for the existing cluster metrics.What this PR does
ControllerFacade#isPrimary()andControllerFacade#isClusterCoordinator(), delegating to the equivalentFlowControllermethods — following the exact pattern of the existingControllerFacade#isClustered()passthrough.ClusterMetricsRegistry, labeledinstanceonly (consistent with the other single-value cluster gauges):cluster_is_primary_nodeandcluster_is_cluster_coordinator.PrometheusMetricsUtil#createClusterMetricsandStandardNiFiServiceFacade#populateFlowMetricsalongside the existing cluster metrics.TestFlowResource's fixture and sample-count assertions for the two new metrics.This lets a dashboard mark the Primary Node and Cluster Coordinator directly from the same Prometheus scrape already used for the rest of the cluster-overview table, without a separate REST API call/join. Also useful for alerting (e.g. "no node in this cluster is reporting as coordinator" indicates a leader-election problem).
Tests
./mvnw -pl nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api -am -P contrib-check clean install— 0 Checkstyle violations, RAT license check passed, 642/642 tests passed, BUILD SUCCESSVerification
Build
./mvnw clean install -P contrib-check(scoped tonifi-framework-bundle/nifi-framework/nifi-web/nifi-web-apiand its dependencies via-pl ... -am; full reactor build not run locally, deferred to CI) — 0 Checkstyle violations, RAT license check passed, 642/642 tests passed, BUILD SUCCESSmaven.compiler.releasetargets 21Licensing
Documentation