🚀 Just launched v1 of managed rate limiting service
Support us with your feedback and questions on Product Hunt or Discord
The aperture-js SDK provides an easy way to integrate your JavaScript
applications with FluxNinja Aperture.
It allows flow control functionality on fine-grained features inside service
code.
Refer documentation for more details.
Aperture provides rate limiting, caching, and api quota management to effectively manage the load on your application. With its unique approach that separates application code from rate limiting code/infra, it can effectively handle various use cases of rate limiting such as:
- Enforcing or complying with
- Rate-limits based on no. of requests per second
- Per-user rate-limits based on consumed tokens (e.g. quota mangement for GPT APIs)
- Rate-limits based on user subscription plans
- Rate-limits based on token-bucket algorithm
- Fine-grained rate-limits configured via policies UI or yaml
- Prioritizing requests based on custom labels (e.g. free user vs paid user)
- Enforcing adaptive rate-limits based on concurrency and available system capacity
- Managing load on your databases or self-hosted services such as Mistral, CodeLlama, etc.
- Caching API responses to avoid high API or Cloud cost and improve time to response
- Updating rate-limiting policies from UI without changing the code
- Monitoring workload for your internal or external services
Are we missing your use case for FluxNinja Aperture? Share with us on Discord
Run the command below to install the SDK:
npm install @fluxninja/aperture-jsThe next step is to create an Aperture Client instance, for which, the address of the organization created in Aperture Cloud and API key are needed. You can locate both these details by clicking on the Aperture tab in the sidebar menu of Aperture Cloud.
import { ApertureClient } from "@fluxninja/aperture-js";
// Create aperture client
export const apertureClient = new ApertureClient({
address: "ORGANIZATION.app.fluxninja.com:443",
apiKey: "API_KEY",
});The created instance can then be used to start a flow:
async function handleRequestRateLimit(req: Request, res: Response) {
// Start a flow by passing control point and business labels
const flow = await apertureClient.startFlow("awesomeFeature", {
labels: {
limit_key: "some_user_id",
},
grpcCallOptions: {
deadline: Date.now() + 300, // ms
},
});
if (flow.shouldRun()) {
// Add business logic to process incoming request
console.log("Request accepted. Processing...");
const resString = "foo";
res.send({ message: resString });
} else {
console.log("Request rate-limited. Try again later.");
// Handle flow rejection
flow.setStatus(FlowStatus.Error);
res.status(429).send({ message: "Too many requests" });
}
flow.end();
}The above code snippet is making startFlow calls to Aperture. For this call,
it is important to specify the control point (awesomeFeature in the example)
and business labels that will be aligned with the policy created in Aperture
Cloud. For request prioritization use cases, it's important to set a higher gRPC
deadline. This parameter specifies the maximum duration a request can remain in
the queue. For each flow that is started, a shouldRun decision is made,
determining whether to allow the request into the system or to rate limit it. In
this example, we only see log returns, but in a production environment, actual
business logic can be executed when a request is allowed. It is important to
make the end call made after processing each request, to send telemetry data
that would provide granular visibility for each flow.
For more context on using the Aperture JavaScript SDK to set feature control points, refer to the example app available in the repository.