What happened to Anyscale Endpoints?

Anyscale sunset the self-serve multi-tenant Endpoints API on August 1, 2024. LLM serving moved into the fully-managed Anyscale Platform, which is positioned at enterprise customers and runs on a different commercial model (annual contracts, dedicated deployments, sales conversations). The self-serve $0.10–$0.50/M-token API many developers were using is gone.

Why not just use the Anyscale Platform?

The Anyscale Platform is built for teams that need Ray-based distributed training + serving and can commit to an enterprise relationship. For teams that just need a private OpenAI-compatible LLM endpoint without operating Ray clusters or signing annual contracts, it's overkill. Auxen sits at the self-serve / startup tier — provision an endpoint in minutes, pay per minute, cancel any time. No sales calls, no annual minimums.

Are there other Anyscale Endpoints alternatives I should consider?

Yes. Together AI and Fireworks AI both offer per-token serverless inference on open-source LLMs and are the most direct shape replacement for Anyscale's old Endpoints product. RunPod is the GPU-rental alternative if you want to run your own inference server. Auxen is the dedicated-endpoint alternative if you want a managed private endpoint billed per minute instead of per token.

Auxen vs Anyscale

Anyscale sunset Endpoints. Auxen is the migration for displaced users.

Q: Is Auxen a direct replacement for Anyscale Endpoints?

For most former Endpoints workloads — yes. Both expose OpenAI-compatible /v1/chat/completions. Auxen carries equivalent open-source models (Llama 3.1, Qwen, Mistral, Gemma, Mixtral, Phi, Command R). The main shift: Auxen bills per minute of dedicated instance runtime instead of per token. If your traffic was steady (the typical SaaS or agent workload), the math is usually better at Auxen.

Q: What does migrating from Anyscale Endpoints to Auxen look like?

Provision an Auxen instance in the dashboard with your model of choice, copy the instance API key, and change the base URL + API key in your existing OpenAI-compatible client. Code changes are minimal. The model menu and request format are familiar. Streaming and tool calling work the same. Most former Anyscale Endpoints users complete migration in under an hour.

Anyscale's self-serve multi-tenant Endpoints API shut down on August 1, 2024. LLM serving moved into the fully-managed Anyscale Platform, which is positioned at enterprise customers on annual contracts. If you were paying $0.10–$0.50/M tokens for open-source LLMs and don't want to enter sales conversations to get an endpoint back, Auxen is the direct replacement at the self-serve tier.

Try Auxen — $10 to start How it works

At a glance

Dimension	Auxen	Anyscale
Status of the product	Live and self-serve. Provision an instance in the dashboard, get an OpenAI-compatible endpoint in minutes.	Self-serve multi-tenant Endpoints API sunset August 1, 2024. LLM serving now requires the full Anyscale Platform (enterprise, contracted).
How you get started	Sign up, top up $10, pick a model, deploy. No sales conversation. Cancel any time.	Contact sales to discuss the Anyscale Platform. Annual contract / dedicated deployment scope conversation.
Pricing model	Per-minute of instance runtime. $0.15/hr (3–7B) up to $2.85/hr (70B+). Pause to stop billing.	Custom enterprise pricing through sales. Former Endpoints was per-token roughly $0.10–$0.50/M.
Catalog (open-source LLMs)	Llama 3.1 / 3.2, Qwen 2.5, Mistral, Mistral Nemo, Mixtral, Gemma 2, Phi-3, Command R.	Anyscale Platform supports any Ray-compatible model (effectively all open-source LLMs) when self-deployed. The curated multi-tenant catalog from Endpoints no longer exists.
API surface	OpenAI-compatible /v1/chat/completions. Drop-in for openai-python, Vercel AI SDK, LangChain — swap base URL and key.	Anyscale Platform exposes Ray Serve endpoints. OpenAI compatibility is configurable but is no longer the default low-friction product surface Endpoints used to be.
Privacy + tenancy	Dedicated GPU. Your prompts and completions run on hardware exclusively yours for the runtime billed.	Anyscale Platform deploys dedicated clusters in your cloud account or theirs. Strong tenancy story, but only via enterprise engagement.
Programmatic lifecycle (MCP)	Full instance lifecycle exposed over MCP — auxen_provision_model, auxen_pause_instance, auxen_set_schedule, auxen_destroy_instance, etc. An agent can self-operate the model without a human. OAuth 2.1 + PKCE. Listed on registry.modelcontextprotocol.io.	No first-party MCP integration. Anyscale's focus is general Ray-based ML rather than agent-specific protocols.
Operational model	Managed endpoint. You don't operate Ray, Kubernetes, or any inference server. Auxen handles uptime and the underlying GPU.	Anyscale Platform is a managed Ray platform — you still configure Ray Serve deployments, scaling policies, model loading. More powerful, more operational surface area.
Best for	Former Endpoints users who want a self-serve OpenAI-compatible LLM endpoint back. Steady traffic on one model. Predictable monthly cost.	Teams with significant Ray investment, dedicated ML platform engineering capacity, or complex multi-model serving needs that justify an enterprise platform engagement.

Anyscale description: Anyscale Platform (anyscale.com) — enterprise Ray-based ML platform. The self-serve multi-tenant Endpoints API was sunset on August 1, 2024; LLM serving now requires the full Platform engagement (annual contracts, dedicated deployments, sales conversations).

Auxen's distinctive axis: programmatic lifecycle control

Pricing shape, model catalog, and latency are real dimensions to compare — but they aren't where Auxen's unique fit lives. The axis the comparison turns on is programmatic lifecycle control: an agent operates the whole instance lifecycle over MCP. auxen_provision_model spins up a private, single-tenant instance. auxen_pause_instance and auxen_set_schedule manage runtime. auxen_destroy_instance stops the meter when the task is done. Per-token serverless APIs cannot structurally offer this — there is no instance for the customer to operate. If your workload is agent-driven and benefits from a private, programmable model for the duration of a task, Auxen wins on autonomy + privacy regardless of whether it wins on raw $/token (often it doesn't, and our pages say so).

What actually happened to Anyscale Endpoints

Anyscale announced on August 1, 2024 that the multi-tenant self-serve Endpoints API — the $0.10–$0.50 per million tokens product that many developers were using — would be shut down. LLM serving moved into the fully-managed Anyscale Platform, which is positioned at enterprise customers and runs on annual contracts. The reasoning was reasonable from Anyscale's side (Ray's strengths are at the platform layer, not the per-token API layer), but it left a real gap: developers who had built on Endpoints' specific shape — OpenAI-compatible, self-serve, no contract — needed somewhere to go.

The migration shape Auxen offers

Auxen takes the same general posture Endpoints did — self-serve, OpenAI-compatible, no contract required — but trades per-token billing for per-minute dedicated billing. For most former Endpoints workloads (a SaaS app or agent loop sending steady traffic to one model), the per-minute math is competitive or better, because the per-token Endpoints rate already assumed you'd accumulate enough tokens to make their margin work. The migration code change is minimal: change base URL, change API key, keep the rest of the openai-python or LangChain integration as-is.

When the Anyscale Platform is still the right answer

If your workload was never really about LLM Endpoints — if you used Anyscale for distributed training, large-scale data processing, Ray Tune for hyperparameter search, or multi-model serving with complex routing — the Anyscale Platform is still the right tool. Auxen is the right answer when you specifically needed the Endpoints product: a single OpenAI-compatible LLM endpoint, self-serve. If your needs go beyond that, evaluate the Platform or stay with whatever Ray infrastructure you already have.

Other Endpoints alternatives worth knowing about

Auxen isn't the only Endpoints replacement. Together AI and Fireworks AI both offer per-token serverless inference on open-source LLMs and are the closest shape match to what Endpoints used to be. If your traffic is sporadic enough that per-token still beats dedicated-hourly, they're worth considering. RunPod is the GPU-rental alternative if you want to operate your own inference server. Auxen's positioning is the dedicated-endpoint slot — managed, OpenAI-compatible, billed per minute.

Which one is right for you?

Pick Auxen if

✓You were using Anyscale Endpoints for an OpenAI-compatible LLM endpoint and want a direct self-serve replacement
✓Your traffic is steady on one model (not bursty across many)
✓You want predictable monthly cost decoupled from token usage
✓You don't want to enter a sales conversation or sign an annual contract
✓You need first-class MCP support for agent workloads
✓You want a managed endpoint without operating Ray clusters

Pick Anyscale if

·Your workload uses Ray for distributed training, hyperparameter search, or data processing — beyond just LLM serving
·You have an existing Ray investment and want to consolidate on one platform
·You need complex multi-model routing or custom inference graphs Ray Serve handles well
·You have a dedicated ML platform team and want maximum operational control
·Your scale justifies an enterprise engagement and you prefer one vendor for the full ML stack

FAQ

Is Anyscale Endpoints really gone?

The self-serve multi-tenant Endpoints API shut down August 1, 2024. Anyscale's LLM serving capabilities still exist, but they're now part of the fully-managed Anyscale Platform — enterprise tier, sales-led. The $0.10–$0.50/M-token self-serve API isn't coming back in that shape.

Why didn't Anyscale just keep Endpoints running?

Anyscale's positioning is around Ray and platform-layer ML infrastructure. Per-token serverless inference at consumer prices was a different business that didn't compound with the rest of their offering. Sunsetting it focused engineering effort on the Platform.

Is Auxen a direct replacement for Anyscale Endpoints?

Yes for most workloads. Both expose OpenAI-compatible /v1/chat/completions. Auxen carries the same open-source models Endpoints was popular for (Llama 3.1, Mistral, Mixtral). The pricing model differs: Auxen bills per minute of dedicated instance time instead of per token, which is usually cheaper if your traffic is steady.

What does migrating from Anyscale Endpoints to Auxen actually look like?

Provision an Auxen instance with your model of choice in the dashboard. Copy the instance API key. Change base URL and API key in your existing OpenAI-compatible client. Code changes are minimal. The model menu and request format are familiar. Most teams complete migration in under an hour.

What if I need fine-tuning, which Endpoints offered?

Auxen's fine-tuning is on the roadmap but not active today. Persona Studio (managed system-prompt + knowledge-base customization on top of any catalog model) is the customization layer currently available. If weight-level fine-tuning is a hard requirement now, Together AI or Fireworks AI offer hosted fine-tuning pipelines; RunPod is the option if you want full control.

Are there other Anyscale Endpoints alternatives?

Yes — Together AI and Fireworks AI are the closest shape replacements if you want per-token serverless inference on open-source LLMs. RunPod is the option if you want raw GPU rental and your own inference server. Auxen is the dedicated-endpoint option: managed, OpenAI-compatible, billed per minute.

See if Auxen fits your workload.

$10 to start. No subscription. Deploy a private model in minutes and see the API surface yourself.

Start with $10 Agent quickstart

Need to deploy something Auxen doesn't support yet? Tell us.

Competitor pricing and product positioning shift quickly. Facts on this page last verified 2026-05-30 against each provider's public docs. If a number looks stale, let us know and we'll fix it.