GPU Templates
Deploy popular AI frameworks and environments without manually configuring every dependency.
InstantGPU templates provide pre-configured environments that help you start GPU workloads faster. Instead of spending hours resolving CUDA drivers, cuDNN dependencies, Python wheels, and framework toolchains, select a verified template and start running experiments in seconds.
Available Templates #
InstantGPU maintains optimized, turn-key templates for the world's most popular AI frameworks, inference servers, and creative toolkits:
Production-grade PyTorch with CUDA 12.x, torchvision, torchaudio, TensorBoard, and JupyterLab pre-installed.
Google TensorFlow with Keras, XLA GPU compiler, TensorRT integration, and distributed training utilities.
Full Hugging Face stack: Transformers, Diffusers, Accelerate, Datasets, PEFT, and TRL for quick fine-tuning.
High-throughput, low-latency LLM serving engine with PagedAttention and OpenAI-compatible HTTP API server.
Modern, modular visual node-graph interface for designing advanced SDXL, Flux, ControlNet, and video workflows.
Optimized inference and fine-tuning runtime for Meta Llama 3, 3.1, and 3.3 models with quantization support.
Turn-key image synthesis suite with Automatic1111 WebUI, Fooocus, SDXL base, and high-performance xFormers.
Ultra-fast automatic speech recognition (ASR) using faster-whisper and WhisperX with word-level timestamps.
Clean Ubuntu Linux environments with selectable CUDA 11.8, 12.1, or 12.4 toolkits, cuDNN, CMake, and build tools.
Deploy a Template #
Launching any template on InstantGPU follows a straightforward 5-step deployment flow:
vLLM, PyTorch, ComfyUI). Click on your preferred card to open the template configuration modal.
Why Use Templates? #
Configuring deep learning stacks manually can lead to version mismatches, broken dynamic libraries, and lost engineering hours. InstantGPU templates eliminate these bottlenecks:
Framework Installation
No manual pip or conda compiling. All core frameworks (PyTorch, TensorFlow, JAX) are baked into optimized root layers with verified wheel packages.
Dependency Configuration
FlashAttention, bitsandbytes, xFormers, and Triton kernels are pre-compiled and tested for zero-runtime crash execution.
Runtime Setup
Turn-key web interfaces, JupyterLab workspaces, and REST endpoints start automatically upon container boot.
CUDA Configuration
NVIDIA display drivers, CUDA toolkits, and cuDNN libraries are perfectly matched to underlying GPU microarchitectures.
Model Deployment
One-click deployment with pre-configured Hugging Face model weight caches to minimize cold starts.
Development Environments
Connect seamlessly from VS Code Remote SSH, PyCharm, Cursor, or your local terminal with full hardware telemetry.
Custom Environment #
If a suitable template is not available, deploy your own Docker image and configure the environment according to your application's requirements. InstantGPU supports any OCI-compliant container from Docker Hub, GitHub Container Registry (GHCR), or private container registries.
# Start with NVIDIA CUDA official base image
FROM nvidia/cuda:12.4.1-devel-ubuntu22.04
# Install system dependencies and Python 3.11
RUN apt-get update && apt-get install -y python3-pip python3-dev git curl \
&& rm -rf /var/lib/apt/lists/*
# Install custom AI packages
RUN pip install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
RUN pip install --no-cache-dir transformers accelerate vllm fastapi uvicorn
# Expose server port and run inference gateway
WORKDIR /app
COPY . /app
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]