What happened to Anyscale Endpoints?

Anyscale sunset the self-serve multi-tenant Endpoints API on August 1, 2024. LLM serving moved into the fully-managed Anyscale Platform, which is positioned at enterprise customers and runs on a different commercial model (annual contracts, dedicated deployments, sales conversations). The self-serve $0.10–$0.50/M-token API many developers were using is gone.

Why not just use the Anyscale Platform?

The Anyscale Platform is built for teams that need Ray-based distributed training + serving and can commit to an enterprise relationship. For teams that just need a private OpenAI-compatible LLM endpoint without operating Ray clusters or signing annual contracts, it's overkill. Auxen sits at the self-serve / startup tier — deploy an endpoint in minutes, pay per minute while it's active, delete it any time. No sales calls, no annual minimums.

Are there other Anyscale Endpoints alternatives I should consider?

Yes. Together AI and Fireworks AI both offer per-token serverless inference on open-source LLMs and are the most direct shape replacement for Anyscale's old Endpoints product. RunPod is the GPU-rental alternative if you want to run your own inference server. Auxen is the private-endpoint alternative if you want a managed, single-tenant endpoint billed per minute of active time instead of per token.

Auxen vs Anyscale

Anyscale sunset Endpoints. Auxen is the migration for displaced users.

Q: Is Auxen a direct replacement for Anyscale Endpoints?

For most former Endpoints workloads — yes. Both expose OpenAI-compatible /v1/chat/completions. Auxen carries equivalent open-weight models across four tiers (Fast, Balanced, Powerful, Maximum). The main shift: Auxen bills per minute of active time instead of per token. Where the two diverge is shape — Auxen is a self-serve, single-tenant private endpoint you deploy in minutes with no shared inference or third-party routing, whereas the Anyscale Platform is enterprise, sales-led, and contract-based.

Anyscale's self-serve multi-tenant Endpoints API shut down on August 1, 2024. LLM serving moved into the fully-managed Anyscale Platform, which is positioned at enterprise customers on annual contracts. If you ran open-source LLMs on a shared, multi-tenant API and don't want to enter sales conversations to get an endpoint back, Auxen is the direct replacement at the self-serve tier — a private, single-tenant model with no shared inference and no third-party data routing.

Try Auxen — $10 to start How it works

At a glance

Dimension	Auxen	Anyscale
Status of the product	Live and self-serve. Deploy a model from the dashboard, get an OpenAI-compatible endpoint in minutes.	Self-serve multi-tenant Endpoints API sunset August 1, 2024. LLM serving now requires the full Anyscale Platform (enterprise, contracted).
How you get started	Sign up, top up $10, pick a model, deploy. No sales conversation. Delete any time.	Contact sales to discuss the Anyscale Platform. Annual contract / dedicated deployment scope conversation.
Pricing model	Per minute of active time, $0.15–$2.25/hr by model tier (Fast to Maximum). Pause to stop billing.	Custom enterprise pricing through sales. Former Endpoints was per-token roughly $0.10–$0.50/M.
Catalog (open-source LLMs)	Curated open-weight models in four tiers — Fast, Balanced, Powerful, Maximum.	Anyscale Platform supports any Ray-compatible model (effectively all open-source LLMs) when self-deployed. The curated multi-tenant catalog from Endpoints no longer exists.
API surface	OpenAI-compatible /v1/chat/completions. Drop-in for openai-python, Vercel AI SDK, LangChain — swap base URL and key.	Anyscale Platform exposes Ray Serve endpoints. OpenAI compatibility is configurable but is no longer the default low-friction product surface Endpoints used to be.
Privacy + tenancy	Single-tenant deployment. Your prompts and completions are answered by a model that serves only you.	Anyscale Platform deploys dedicated clusters in your cloud account or theirs. Strong tenancy story, but only via enterprise engagement.
Lifecycle control (MCP)	Full deployment lifecycle exposed over MCP — auxen_deploy_model, auxen_set_throughput, auxen_delete_deployment, etc. Control your own private, single-tenant deployment programmatically, and delete it to erase the data when you're done. OAuth 2.1 + PKCE. Listed on registry.modelcontextprotocol.io.	No first-party MCP integration. Anyscale's focus is general Ray-based ML rather than agent-specific protocols.
Operational model	Managed endpoint. You don't operate Ray, Kubernetes, or any inference server. Auxen handles uptime and everything underneath.	Anyscale Platform is a managed Ray platform — you still configure Ray Serve deployments, scaling policies, model loading. More powerful, more operational surface area.
Best for	Former Endpoints users who want a self-serve OpenAI-compatible LLM endpoint back. Steady traffic on one model. Predictable monthly cost.	Teams with significant Ray investment, dedicated ML platform engineering capacity, or complex multi-model serving needs that justify an enterprise platform engagement.

Anyscale description: Anyscale Platform (anyscale.com) — enterprise Ray-based ML platform. The self-serve multi-tenant Endpoints API was sunset on August 1, 2024; LLM serving now requires the full Platform engagement (annual contracts, dedicated deployments, sales conversations).

Auxen's distinctive axis: single-tenant privacy

Pricing shape, model catalog, and latency are real dimensions to compare — but they aren't where Auxen's unique fit lives. The axis the comparison turns on is single-tenant isolation. Endpoints was a shared, multi-tenant API: your prompts and completions moved through inference infrastructure serving every other customer at once. Auxen is the opposite — a private model in a deployment exclusively yours while it's active. No shared inference, no third-party data routing, and when you delete the deployment the data goes with it. And you get there self-serve: sign up, pick a model, deploy. Anyscale sunset self-serve Endpoints and now routes LLM serving through the enterprise, sales-led Platform; Auxen keeps the self-serve path but makes it single-tenant. Lifecycle control over MCP (auxen_deploy_model, auxen_set_throughput, auxen_delete_deployment) is a capability layered on top — a way to operate your own private deployment programmatically, not a claim about agent autonomy.

What actually happened to Anyscale Endpoints

Anyscale announced on August 1, 2024 that the multi-tenant self-serve Endpoints API — the $0.10–$0.50 per million tokens product that many developers were using — would be shut down. LLM serving moved into the fully-managed Anyscale Platform, which is positioned at enterprise customers and runs on annual contracts. The reasoning was reasonable from Anyscale's side (Ray's strengths are at the platform layer, not the per-token API layer), but it left a real gap: developers who had built on Endpoints' specific shape — OpenAI-compatible, self-serve, no contract — needed somewhere to go.

The migration shape Auxen offers

Auxen takes the same general posture Endpoints did — self-serve, OpenAI-compatible, no contract required — but replaces shared multi-tenant serving with a private, single-tenant deployment, and per-token billing with billing per minute of active time. The important difference isn't the billing unit; it's the tenancy. On Endpoints your traffic shared inference infrastructure with everyone else; on Auxen it runs on a private model that's yours alone while it's active, with no third-party data routing and data erased when you delete the deployment. The migration code change is minimal: change base URL, change API key, keep the rest of the openai-python or LangChain integration as-is.

When the Anyscale Platform is still the right answer

If your workload was never really about LLM Endpoints — if you used Anyscale for distributed training, large-scale data processing, Ray Tune for hyperparameter search, or multi-model serving with complex routing — the Anyscale Platform is still the right tool. Auxen is the right answer when you specifically needed the Endpoints product: a single OpenAI-compatible LLM endpoint, self-serve. If your needs go beyond that, evaluate the Platform or stay with whatever Ray infrastructure you already have.

Other Endpoints alternatives worth knowing about

Auxen isn't the only Endpoints replacement. Together AI and Fireworks AI both offer per-token serverless inference on open-source LLMs and are the closest shape match to what Endpoints used to be. If your traffic is sporadic enough that per-token still beats a per-minute rate, they're worth considering. RunPod is the GPU-rental alternative if you want to operate your own inference server. Auxen's positioning is the private-endpoint slot — managed, single-tenant, OpenAI-compatible, billed per minute while it's active.

Which one is right for you?

Pick Auxen if

✓You were using Anyscale Endpoints for an OpenAI-compatible LLM endpoint and want a direct self-serve replacement
✓Your traffic is steady on one model (not bursty across many)
✓You want a private, single-tenant model with no shared inference and no third-party data routing
✓You don't want to enter a sales conversation or sign an annual contract
✓You want programmatic control of your own deployment lifecycle over MCP
✓You want a managed endpoint without operating Ray clusters

Pick Anyscale if

·Your workload uses Ray for distributed training, hyperparameter search, or data processing — beyond just LLM serving
·You have an existing Ray investment and want to consolidate on one platform
·You need complex multi-model routing or custom inference graphs Ray Serve handles well
·You have a dedicated ML platform team and want maximum operational control
·Your scale justifies an enterprise engagement and you prefer one vendor for the full ML stack

FAQ

Is Anyscale Endpoints really gone?

The self-serve multi-tenant Endpoints API shut down August 1, 2024. Anyscale's LLM serving capabilities still exist, but they're now part of the fully-managed Anyscale Platform — enterprise tier, sales-led. The $0.10–$0.50/M-token self-serve API isn't coming back in that shape.

Why didn't Anyscale just keep Endpoints running?

Anyscale's positioning is around Ray and platform-layer ML infrastructure. Per-token serverless inference at consumer prices was a different business that didn't compound with the rest of their offering. Sunsetting it focused engineering effort on the Platform.

Is Auxen a direct replacement for Anyscale Endpoints?

Yes for most workloads. Both expose OpenAI-compatible /v1/chat/completions. Auxen carries the same open-weight model families Endpoints was popular for, plus newer releases. The key difference is tenancy: Endpoints was a shared multi-tenant API, whereas Auxen runs your model in a private, single-tenant deployment with no shared inference and no third-party data routing. Billing also differs — per minute of active time rather than per token.

What does migrating from Anyscale Endpoints to Auxen actually look like?

Deploy your model of choice from the Auxen dashboard (or POST /v1/deploy). Copy the deployment's API key. Change base URL and API key in your existing OpenAI-compatible client. Code changes are minimal. The model menu and request format are familiar. Most teams complete migration in under an hour.

What if I need fine-tuning, which Endpoints offered?

Auxen's fine-tuning is on the roadmap but not active today. Persona Studio (managed system-prompt + knowledge-base customization on top of any catalog model) is the customization layer currently available. If weight-level fine-tuning is a hard requirement now, Together AI or Fireworks AI offer hosted fine-tuning pipelines; RunPod is the option if you want full control.

Are there other Anyscale Endpoints alternatives?

Yes — Together AI and Fireworks AI are the closest shape replacements if you want per-token serverless inference on open-source LLMs. RunPod is the option if you want raw GPU rental and your own inference server. Auxen is the private-endpoint option: managed, single-tenant, OpenAI-compatible, billed per minute while it's active.

See if Auxen fits your workload.

$10 to start. No subscription, billed per minute while it's active. Deploy a private model in minutes and see the API surface yourself.

Start with $10 Agent quickstart

Need to deploy something Auxen doesn't support yet? Tell us.

Competitor pricing and product positioning shift quickly. Facts on this page last verified 2026-05-30 against each provider's public docs. If a number looks stale, let us know and we'll fix it.