Wrapper for managing high volume of duplicate idempotent asynchronous requests in JavaScript & TypeScript.
When you have a high volume of duplicate, overlapping, idempotent requests. If your application is making a number of requests with the same inputs, and the requests are idempotent, caching can be helpful to avoid unnecessary computation. If these requests are long-running and many duplicate requests are made before a response is cached for that request, there can be a lot of wasted computation.
Here is an example timeline only using caching. Requests 1, 2, 3, and 4 all have the same input and will receive the same output.
time
request 1
-> cache miss
-> make call to external service
|
| request 2
| -> cache miss
| -> make call to external service
| |
| | request 2
| | -> cache miss
| | -> make call to external service
| | |
| | |
| | |
-> request 1 resolved, update cache
| |
| | request 4
| | -> use cached response from request 1
| |
-> request 2 resolved, update cache
|
|
-> request 3 resolved, update cache
When requests 2 and 3 are made, request has not finished, so the cache will not be populated. There are two unfortuante outcomes of this approach:
- Requests 2 and 3 will waste resources by performing the same expensive and long-running request 1 is triggering.
- When request 1 is fulfilled, there is enough information to fulfill requests 2 and 3, but instead requests 2 and 3 will not resolve until they get a response from the external service.
Cachallel optimizes for overlapping, duplicate requests by letting subsequent requests latch onto an existing Promise from a previous request.
request 1
-> cache miss
-> no pending Promise for this input
-> make call to external service
|
| request 2
| -> cache miss
| <--- attach to request 1's Promise
|
| request 3
| -> cache miss
| <--- attach to request 1's Promise
|
|
|
-> request 1 resolved
-> update cache
-> request 2 resolved
-> request 3 resolved
request 4
-> use cached response from request 1
import axios from "axios";
import { RedisClient } from "redis";
import { RequestManager, RedisCacheManager } from "cachallel";
async function getResourceFromSlowApi(
id: number,
): Promise<{ id: number; name: string }> {
return await axios.get(`external.service/resource/${id}`);
}
const redisClient = new RedisClient({ host: "127.0.0.1" });
const slowApiManager = new RequestManager(
getResourceFromSlowApi,
new RedisCacheManager(redisClient),
);
export default async function call(id: number) {
return await slowApiManager.call(id);
}