Bug: Inefficient In-Memory Cache Pruning Causes Event Loop Blocking

[TanCodeX]

Summary

The application uses a custom in-memory cache (userCache) with a manual pruning mechanism. The pruneUserCache() function is executed on cache misses and iterates over the entire Map to remove expired entries, resulting in an O(N) operation. Under high traffic or when the cache grows large, this can block the Node.js event loop and degrade application performance.

Affected File

• server.js

Description

The application maintains a custom userCache and periodically removes expired entries by scanning the entire cache.

Since pruneUserCache() is invoked on cache misses, every cache miss may trigger a full traversal of the cache:

• Time Complexity: O(N)
• Event Loop Impact: High as cache size increases
• Scalability: Poor under heavy load

This approach does not scale well because Node.js executes JavaScript on a single-threaded event loop. Large synchronous iterations can delay request processing and increase response latency.

Steps to Reproduce

1. Populate userCache with a large number of entries.
2. Generate repeated cache misses.
3. Observe that pruneUserCache() iterates over the entire cache on each miss.
4. Monitor CPU usage and request latency under load.
5. Notice increased event loop blocking as the cache grows.

Expected Behavior

Cache eviction should occur with minimal overhead and should not require scanning the entire cache on cache misses.

Actual Behavior

Every cache miss may trigger an O(N) cache pruning operation, causing unnecessary CPU usage and event loop blocking.

Performance Impact

• Increased request latency under load.
• Event loop blocking due to synchronous cache traversal.
• Reduced throughput as cache size grows.
• Poor scalability for applications with frequent cache misses.

Recommendation

Replace the custom cache implementation with a well-tested Least Recently Used (LRU) cache library such as lru-cache, which provides efficient cache eviction and expiration mechanisms.

Example:

const { LRUCache } = require('lru-cache');

const userCache = new LRUCache({
  max: 1000,
  ttl: 1000 * 60 * 5, // 5 minutes
});

Benefits include:

• Efficient cache eviction.
• Built-in TTL support.
• No manual pruning logic.
• Better performance under high load.
• Reduced event loop blocking.

References

• lru-cache Documentation: https://github.com/isaacs/node-lru-cache
• Node.js Guide: Don't Block the Event Loop

[github-actions]

Thank you for opening an issue.

Please review our CONTRIBUTING.md guidelines before getting started.

A maintainer will review this issue shortly. If you would like to work on the issue yourself, please leave a comment requesting assignment.

Issue Assignment Policy

• The issue creator is given the first priority to work on their issue.
• If the creator declines or does not request assignment, the issue will be assigned to others in the order of requests received.
• Please do not submit PRs or begin work until you are officially assigned.

Note: This project is currently maintained by a solo maintainer, so reviews, assignments, and responses may sometimes take a little time. Thanks for your patience.

[TanCodeX]

@jagdish-15 Please assign me this under gssoc.

[jagdish-15]

I don't think this is necessary for our use case. The cache is intentionally bounded (1000 entries), and pruning only runs once that limit is exceeded. Given the small cache size and short TTL, the current implementation keeps the logic simple without introducing an additional dependency. I don't think the overhead here is significant enough to justify replacing it with an LRU cache.