Fix orphan blob accumulation on firmware upgrade

[weebl2000]

Fixes #1519

Related: #1495

What's this about?

PR #1495 fixed the blob leak going forward, but devices upgrading from older firmware (like v1.11.0) still have thousands of orphan blobs sitting in /bl/ eating up storage space.

This adds a one-time cleanup that runs on first boot after upgrade - it just walks through /bl/, checks each blob against the contacts list, and deletes any that don't belong to an actual contact. A marker file prevents it from running again.

Only affects ESP32 and RP2040 (the platforms that use the /bl/ directory).

This code can probably be removed after a few releases once most users have upgraded past this version.

Note

I also experimented with an LRU approach that caps blob count and evicts oldest files periodically - works well as an ongoing safety net, but this simpler approach should be enough for the upgrade cleanup.

Testing

• Flash on a device with existing orphan blobs
• Check cleanup runs once and cleans up the orphans
• Reboot and verify it doesn't run again
• Make sure legit contact blobs are still there

---

Build firmware: Build from this branch

[oltaco]

Good call, I should have done this so thank you 😁

Couple of things, does this belong in DataStore rather than MyMesh? Then you can just call remove directly.

Also it should probably check for the full key to avoid accidentally overwriting blobs with the same prefix.

I'm on phone so my apologies if I've misunderstood the code.

[weebl2000]

Good call, I should have done this so thank you 😁

Couple of things, does this belong in DataStore rather than MyMesh? Then you can just call remove directly.

Also it should probably check for the full key to avoid accidentally overwriting blobs with the same prefix.

I'm on phone so my apologies if I've misunderstood the code.

Yeah, DataStore is probably cleaner. Changed that.

With regards to checking full key, the filename is only 8-bytes (16 hex chars). If we would want to check full key we would need to read the files for the advert info so that might be overkill. Chance for collision is very low though with 8 bytes.

[weebl2000]

Ran it on my heltec v4 and all my contacts are still there, but disk usage was reduced by over 1 MB.

[syssi]

I would be happy about a cleanup once per reboot.

[syssi]

Tested this approach on a Seeed Xiao S3 WIO (ESP32-S3), env Xiao_S3_WIO_companion_radio_wifi, and it resolves the storage exhaustion behind #1519.

Before cleanup, the device had accumulated a large number of orphan blobs even though #1495 is already merged (the merge stops new leakage but does not remove what was already there). Measured at boot:

FS: total=1404 KB used=1312 KB free=92 KB
total_files=2292   (all /bl advert blobs, ~130 bytes each)

With max 350 contacts there should be at most ~350 blobs, so roughly 1942 were orphans. They held only ~302 KB of logical data but consumed 1312 KB of flash due to SPIFFS small file overhead, leaving only 92 KB free, which is below the ~53 KB the contacts file needs and is what triggers the contact loss.

After the one time cleanup runs (walk /bl, delete every blob whose key matches no current contact):

cleanupOrphanBlobs: removed=2288
FS now used=200 KB free=1204 KB

Storage recovered from 92 KB free to 1204 KB free, and legit contact blobs for the active contacts were kept. I implemented the same idea independently and can confirm the logic is sound. One note from my run: deleting ~2288 files takes a few seconds, so feeding the watchdog during the sweep (yield every N deletes, or chunked passes) is worth keeping. The marker file approach for running it once is sensible.