Bound the in-memory batch buffer to prevent unbounded growth

[Dodothereal]

Bound the in-memory batch buffer to prevent unbounded growth

Summary

This PR addresses issue #4 by implementing a bound on the in-memory batch buffer to prevent unbounded growth.

Changes

• internal/pipeline/batch.go:

  • Added a max field to the Batch struct to track the maximum size.
  • Added a dropped counter to track the number of points discarded due to size limits.
  • Added NewBatch(max int) *Batch factory function to create batches with a configured maximum size.
  • Updated the Add method to drop the oldest points first when the buffer would exceed its maximum size, thereby preserving the newest data (which is most valuable for monitoring).
  • The Add method now returns the number of points dropped in that call.
  • Added Dropped() int64 and DroppedReset() int64 methods to read and reset the dropped counter.

• internal/app/run.go:

  • Changed batch initialization from &pipeline.Batch{} to pipeline.NewBatch(cfg.BatchMax) to use the new factory.
  • In sendBatch, added logging of dropped points (via batch.DroppedReset()) at INFO level whenever the batch is flushed, providing periodic insight into data loss due to the size limit.

• internal/pipeline/batch_test.go:

  • Rewrote tests to validate the new behavior:
    • Verifies that when the buffer is full, the oldest points are dropped and the newest are retained.
    • Tests the dropped counter correctness and its reset behavior.
    • Tests edge cases (zero or negative buffer size).

Testing

• All unit tests pass, including the new tests for the batch size limiting and dropping policy.
• Manual verification that the application builds and starts successfully.
• The batch size limit is configurable via the KANSHI_BATCH_MAX environment variable (default 100).

Fixes #4

AI assistance disclosure

This contribution was produced by an autonomous AI coding agent (Claude Code) that @Dodothereal operates and monitors. @Dodothereal is accountable for it, will address review feedback promptly, and will close this PR immediately if this kind of contribution is unwelcome in this project. Commits carry an Assisted-by: Claude Code trailer.

[Johnyoat]

Thank you for you submission. Do you understand go or just vibe coded the solutions?

[Johnyoat]

Converted to draft while agent #4 is resolved without code. The current synchronous loop cannot collect while sendBatch retries, and Flush() removes the payload before retrying, so no outage buffer grows. This bounded-buffer change is not needed unless the pipeline becomes concurrent.