ARBR
An open-source self-hosted gateway for AI request routing, governance, and observability
About ARBR
ARBR provides a unified OpenAI-compatible endpoint that connects model providers and selects models for requests using rules, cost guardrails, difficulty-aware selection, and fallback mechanisms. In the request path, it supports budgets, rate limits, output guardrails, prompt injection checks, and a kill switch; it also observes spend, performance, and reliability by provider, model, application, workflow, team, and user, with local dashboard queries or export to OpenTelemetry.
Best for
Best use cases
Unify access to multiple model providers
Select models by request cost and difficulty
Enforce budget and security rules in the request path
Analyze usage by team and user
Features
Intelligent model selection
Selects a model for each request using rules, cost guardrails, difficulty-aware selection, and fallback mechanisms.
Request-path governance
Enforces budgets, rate limits, output guardrails, prompt injection checks, and a kill switch in the request path.
Usage observability
Observes spend, performance, and reliability by provider, model, application, workflow, team, and user.
Open integrations
Provides an OpenAI-compatible endpoint, embeddings, real-time voice agents, and JavaScript and Python SDKs.
Pros
- Provides an OpenAI-compatible endpoint and supports self-hosting and provider neutrality.
- Budgets, rate limits, and security checks can be enforced in the request path.
- Supports observing spend, performance, and reliability across multiple business dimensions, with export to OpenTelemetry.
Cons
- The fine-tuning dataset feature has not yet been delivered.