Serverless GPU
Run AI workloads without managing traditional GPU servers.
InstantGPU Serverless lets you deploy AI functions and applications while infrastructure handles GPU execution and scaling. You deploy code as a worker, receive a secure HTTPS endpoint, and only pay for the exact execution seconds when requests are running.
How It Works #
The end-to-end serverless lifecycle:
Create a Serverless Deployment #
Follow these 6 steps to launch your serverless endpoint:
1. Select a Template or Custom Container
Choose from pre-configured images for popular architectures (vLLM, Whisper, Stable Diffusion, Ollama) or specify your own Docker repository.
2. Choose Your GPU
Pick the GPU card type (RTX 4090, L40S, A100, H100) and number of GPUs per worker required for your model's parameters and VRAM.
3. Configure Your Model and Runtime
Set environment variables, attach network volumes for persistent weights, configure timeout limits, and specify maximum worker concurrency.
4. Deploy Your Workload
Click Deploy in your dashboard or use the InstantGPU CLI/API. The control plane builds and validates your worker configuration.
5. Receive Your Endpoint
You receive a dedicated, load-balanced HTTPS endpoint URL with synchronous (/runsync) and asynchronous queue (/run) interfaces.
6. Send Requests to Your Endpoint
Post JSON input payloads using Python, cURL, or Node.js. The endpoint automatically boots workers, executes your function, and returns output.
Automatic Scaling #
Serverless infrastructure can scale according to workload demand. This is useful when your traffic changes throughout the day or when you don't want to maintain continuously running GPU instances.
Demand-Driven Bursting
When traffic surges, the autoscaler immediately spawns parallel worker containers across healthy GPU nodes to maintain low response times.
Scale to Zero
When there are no active requests, serverless workloads automatically scale down to zero according to your idle timeout configuration, dropping compute costs to $0.
Custom Containers #
Bring your own Docker image when you need:
Custom Dependencies
Specific Python libraries, proprietary codebases, or specialized compiled packages.
CUDA Configurations
Specific CUDA toolkits, cuDNN, and TensorRT versions optimized for your network.
Specific Libraries
Custom FlashAttention, triton kernels, or hardware-specific quantization packages.
Custom Model Runtimes
Tailored serving engines, bespoke tokenizers, or multi-modal fusion pipelines.
Best For #
InstantGPU Serverless is ideally suited for:
AI Inference
Serving LLMs, vision models, diffusion pipelines, and speech synthesis APIs on demand.
Variable Traffic
Applications with unpredictable traffic spikes or daily quiet hours (e.g. nighttime).
Event-Driven Workloads
Triggered document parsing, image generation webhooks, and asynchronous background jobs.
Production AI APIs & Burst Workloads
SaaS products integrating intelligence features without dedicating full-time GPU clusters.