Problem
Kafka functions (run.kafka) and KEDA consumer-lag scaling (#4051) have no automated tests against a real broker. func cluster create / hack/cluster.sh install no Kafka, so #3956 can't run in CI, and manual testing needs hand-written manifests.
Proposal
Add --kafka ($FUNC_CLUSTER_KAFKA, default off) next to --serving, --eventing, --keda and --tekton:
- install a single-node Kafka broker in a
kafka namespace, reachable at kafka.kafka.svc.cluster.local:9092
- pin its version in
hack/component-versions.json
- optionally a
SASL_PLAINTEXT listener with a fixed test user, to test credentials (TriggerAuthentication)
Open questions
Related: #3956, #4051
Problem
Kafka functions (
run.kafka) and KEDA consumer-lag scaling (#4051) have no automated tests against a real broker.func cluster create/hack/cluster.shinstall no Kafka, so #3956 can't run in CI, and manual testing needs hand-written manifests.Proposal
Add
--kafka($FUNC_CLUSTER_KAFKA, default off) next to--serving,--eventing,--kedaand--tekton:kafkanamespace, reachable atkafka.kafka.svc.cluster.local:9092hack/component-versions.jsonSASL_PLAINTEXTlistener with a fixed test user, to test credentials (TriggerAuthentication)Open questions
apache/kafkain KRaft mode (a single Deployment and Service; small and fast, and it worked for manual testing of Add KEDA Kafka consumer-lag scaling support #4051) or Strimzi (as Kafka functions: integration and E2E tests #3956 suggests; closer to production, heavier on CI runners)?Related: #3956, #4051