whichdevtool?
Menu
StacksDirectory Pricing changes Glossary
VPS and cloud servers App hosting and PaaS Managed databases Transactional email APIs Email marketing and newsletters Domain registrars CDN, DNS and edge security Monitoring, uptime and error tracking Auth and identity Frontend and full-stack frameworks Backend frameworks UI component libraries AI model APIs File and object storage Payments and billing Background jobs, queues and workflows ORMs and database toolkits

Inference provider (open-weight models) · by Groq

GroqCloud

Inference API on Groq's own LPU hardware for open-weight models such as gpt-oss and Qwen, with a free plan and published output speeds.

Visit groq.com ↗ Alternatives Free tier
GroqCloud homepage
§1

Overview

Groq designs its own inference chip, the LPU, and GroqCloud is the API that runs open-weight models on it. The current self-serve catalogue centres on OpenAI's gpt-oss 120B and 20B, Qwen 27B models, Whisper speech recognition and Groq's Compound agent system; Llama 3.1 8B and 3.3 70B are now listed as Enterprise models with sales pricing. The docs publish an output speed per model, such as about 1,000 tokens per second for GPT OSS 20B.

The free plan needs no payment and gives each model its own requests-per-minute, requests-per-day, tokens-per-minute and tokens-per-day caps; gpt-oss-120b, for example, allows 30 requests a minute and 1,000 a day. Exceeding a limit returns HTTP 429. The Developer plan is pay-as-you-go with no subscription fee and has higher limits, Batch processing at half price and Flex processing.

Prompt caching is automatic on the gpt-oss models: cached input is half price and does not count toward rate limits. Before building on a specific model, check its status, since Groq moves models between self-serve and enterprise availability.

§2

Pricing and plans

Free tier

Free plan with per-model limits, e.g. gpt-oss-120b at 30 requests/min, 1,000 requests/day, 8K tokens/min and 200K tokens/day; Whisper at 20 RPM and 2,000 RPD.

At the limit Stops at the limit Requests over a limit get HTTP 429 Too Many Requests until the window resets.

Plan Price What you get
Free $0 Per-model RPM/RPD/TPM/TPD caps
GPT OSS 120B $0.15 in / $0.60 out per 1M About 500 tokens/sec
GPT OSS 20B $0.075 in / $0.30 out per 1M About 1,000 tokens/sec
Whisper Large v3 $0.111 per hour of audio Whisper Large v3 Turbo $0.04 per hour
Llama 3.1 8B / 3.3 70B Contact sales Listed as Enterprise models
Developer Pay as you go Higher limits plus Batch (50% off) and Flex processing

Per 1M tokens in USD, read 2026-09-29 on the GroqDocs models, rate-limit, batch and prompt-caching pages (groq.com/pricing now shows no price table). The free-plan limits quoted are from the rate-limit page table; the page wording around which plan they belong to is ambiguous, and third-party 2026 summaries seen in search results describe them as free-plan limits. Cached input tokens are 50% off on GPT-OSS 20B, 120B and Safeguard 20B and do not count toward rate limits. Batch jobs complete in 24 hours to 7 days at 50% off.

Source: console.groq.com/docs/models · prices as published, excluding tax.

§3

What it does

LPU hardware
Models run on Groq's own chips; the docs list output speeds per model.
Free plan
Per-model request and token caps with no subscription fee.
Batch API
50% off synchronous prices with completion windows from 24 hours to 7 days.
Prompt caching
Cached input 50% off on gpt-oss models and excluded from rate limits.
Speech-to-text
Whisper Large v3 at $0.111 per audio hour, Turbo at $0.04.
OpenAI-compatible API
Rate-limit headers and 429 handling follow familiar conventions.
§4

Strengths and limitations

Strengths

  • A permanent free plan with published per-model limits.
  • Published per-model output speeds help latency planning.
  • Low prices on gpt-oss models, halved again by Batch or caching.

Limitations

  • Small self-serve catalogue; Llama models are now enterprise-only with sales pricing.
  • Free-plan limits are tight for production (e.g. 1,000 requests a day on gpt-oss-120b).
  • Batch and Flex require the Developer plan.
  • Prompt caching currently covers only three gpt-oss models.
§5

Who it suits

Good for

  • Voice and chat apps where response speed matters
  • Free prototypes on open-weight models
  • Transcription with Whisper at low hourly rates

Look elsewhere if

  • Teams that need proprietary frontier models
  • Apps that need a broad catalogue of open models (use a router or a larger inference provider)
§6

Alternatives to GroqCloud

Tool From Free option Stages
Scaleway Generative APIs Inference provider (EU-hosted open-weight models)
Usage-based Trial
Mistral API (Mistral Studio) AI model API (model maker)
Usage-based Free tier
Replicate Model hosting API (image, video, audio and LLMs)
$0.09/hr Trial
Gemini API AI model API (model maker)
Usage-based Free tier
DeepInfra Inference provider (open-weight models)
Usage-based Paid
OpenRouter Model router (unified API)
— Free tier

All AI model APIs →

§7

Questions

Is Groq's API free?

There is a free plan with per-model limits, for example 30 requests per minute and 1,000 requests per day on gpt-oss-120b. Paid use is pay-as-you-go on the Developer plan.

How much does gpt-oss-120b cost on Groq?

$0.15 per 1M input tokens and $0.60 per 1M output tokens on 2026-09-29, with cached input at half price.

Does Groq still offer Llama models?

Llama 3.1 8B and Llama 3.3 70B are listed as Enterprise models with contact-sales pricing in Groq's docs on 2026-09-29.

Sources read for this page